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Abstract 


The academic register of English language has been thought of as a challenging cognitive and 
pedagogical endeavour for non-native university students whether their L2 educational 
context is domestic or international. To facilitate the acquisition of academic language, 
several approaches were proposed over years, chief among which are Computer Assisted 
Language Learning (CALL), and Mobile Assisted Language Learning (MALL). Each of the 
two language learning approaches has been exclusively employed to teach one feature of the 
inseparable academic language lexico-grammar at moderate levels of efficacy, yet the joined 
cognitive forces of the two approaches have not been thoroughly examined in the available 
literature. In this study, a short intervention was designed to teach 30 Saudi university 
students the most frequent 40 general academic vocabulary items, several frequent academic 
phrases, and 1 salient academic language feature, namely hedging to improve the lexico- 
grammar of Saudi university students’ written production. Target vocabulary was directly 
derived from Oxford Phrasal Academic Lexicon (OPAL). CALL used as the main medium of 
instructions, in-class activities, and exercises, and MALL employed to compliment the 
intervention’s activities outside class, and enhance target vocabulary’s entrenchment and 
priming in student’s academic repertoire. Pre-/post-test results of six different variables show 
statistically significant correlation between the use of CALL and MALL together and Saudi 
university students’ academic language gains. A large effect size (d= 1.6) was observed for 
vocabulary-related variables, a medium effect size (d= 1.08) for grammar and structure- 


related variables, and a small effect size (d= .65) for academic voice-related variables. 


Chapter One: Introduction 

1.1 Overview 

Learning happens when learners are offered adequately engaging opportunities where they 
are able to see a learning task as valuable, purposeful, and useful, and perceive themselves as 
capable of doing it (Lee & Egbert 2016). Used well, computers can both support and provide 
engaging learning opportunities to foreign language learners. Egbert (2010) maintains that 
using computers to assist language learning can lead to greater success by differentiating and 
providing access to data and/or interactions that were inaccessible otherwise, and/or affording 
more efficient learning rates in pursuit of language goals and learning objectives. 
Accordingly, and in the light of the growing centrality of technology, computer assisted 
language learning (CALL) has reshaped language pedagogy. This is true because, as 
delineated in Chapelle (2001: 27-43), CALL is placed within six sub-disciplines that integrate 
and complement each other in a cognitively feasible fashion, namely computer-supported 
collaborative learning, artificial intelligence, educational technology, computational 


linguistics, corpus linguistics, and computer-assisted assessment. 


CALL, however, has been notoriously thought of as a language pedagogy panacea, where 
second language acquisition researchers proposed it as an instructional solution, and then 
started searching for a problem that could be solved using it (Clark 1994). Several years later, 
SLA community realized that the sheer focus and devotion to computers in themselves was 
not enough, and realized the necessity for SLA research in CALL contexts (Bangs & Cantos 
2004). Accordingly, CALL was agreed to only replace elements deemed valid by research 
and experience, and as CALL developed, technology encouraged SLA researchers and 


teachers to engage learners in ways never attempted before. 


1.2 Language Teachers and CALL 


The roles of language teachers and language learners changed as CALL technologies 
advanced. Learners are viewed as active participants in their language education rather than 
passive recipients of information. Accordingly, teachers are faced with new demands, namely 
to integrate technology seamlessly into their foreign language classrooms, and to find new 
ways to offer their students meaningful communication that contextualize the introduction of 
linguistic skills. Park and Son (2009) maintain that CALL changes the traditional teacher- 
student roles where teachers become more of facilitators and counselors rather than 
informants and decision makers. Their function in the language classroom shifts from 
transmitting knowledge to providing students with tools to acquire linguistic knowledge, and 
understand the value of what technology has to offer. This may imply that the role of the 
teach in CALL context is even more crucial compared to traditional EFL teaching contexts 
because their role extends itself into creating language learning environments and 
experiences that are meaningful, effective and supportive of using CALL technology. Grupa 


(2004: 636) cites: 


Never before have teachers so urgently needed to know what knowledge 


they want to transmit and for what purpose, to decide what are the more 


or less important aspects of that knowledge, and to commit themselves 


to an educational vision they believe in. 


Computers; hence, did not only change traditional class dynamics, but they also changed how 
materials are designed, and how assessments are conducted where the teacher becomes a 
mediator between computers and language learners throughout the entire educational process. 


That mediation is most significant in the cases when teachers are required to deepen students’ 


understanding of the relationship between text and context “in order to avoid portraying 
multimedia in simplistic ways” (Grupa 2004: 637). The present study employs a student- 
focused design where the teacher/researcher takes a noninvasive instructional role targeted to 
assist students overcome technology-induced anxiety in a challenging EFL environment in a 
Saudi higher education context. As will be elaborated upon in the review of literature, CALL 
in Saudi university education suffers a large number of internal and external constraints that 
prevents its proper implementation, and explains the high level of anxiety exhibited by the 


participants. 


1.3 Research in CALL 


Research on CALL domain generally focuses on the efficacy of learning opportunities 
afforded by technology, the way these technologies are constructed and configured in order 
to provide engaging learning opportunities, and the way language learners can autonomously 
use these technologies in language education beyond traditional class instructions. Chapple 
(2008:589) maintains that research goals on CALL have been shaped by “the imperative for 
knowledge that can be put into practice.” Language teachers need to be informed about 
applied linguists’ findings on the application of technology in L2 pedagogy. Insights 
provided by descriptive research, on the one hand, are of paramount importance to L2 
teachers in order to understand what technology is able to add to language instructions, and to 
identify reasons for and methods through which learners develop different communicative 
competences in ways that do not typically happen in the traditional L2 classroom. Evaluative 
research, on the other hand, sets out to prove the quality of CALL-based education by 
comparing results obtained using technology-based methods to results of learning in a 
traditional classroom. Chapple (2008) maintains that fruitful comparative research 


“investigates the outcomes of two real options for pedagogical tasks...rather than attempting 


to address the intractable issue of computer effectiveness.” To that end, the present study 
shall be fully experimental in nature with a control group to empirically compare learning 


gains of a CALL-based vocabulary intervention, and traditional vocabulary instructions. 


In the present study, a selection of 40 academic vocabulary items, and a number of 
phraseological expressions salient in academic discourse derived from Oxford Phrasal 
Academic Lexicon (OPAL) was taught contextualized in four short essays. Contextualized 
target lexical items was then be explicitly taught using a Web 2.0 flashcard platform, namely 
Quizlet. Web-based in-class exercise was conducted to reinforce the correct collocation and 
grammatical usage on the new lexical items. Target lexical items was then reiterated and 
practiced via mobile phone text messages facilitated by a smart phones mobile application, 
namely Whatsapp, where messages with specific exercises was sent at fixed intervals to 
enhance the enhance the rapid entrenchment of lexical items’ meaning and usage. Employing 
academic pedagogical wordlists to inform a computer assisted vocabulary-learning 
intervention that was paired with out-of-class mobile assisted language learning practice 
made possible via smart phones’ applications promise an interdependent, thorough and 
potentially highly effective application of CALL for English for Academic Purposes 


vocabulary instructions. 


Chapter two: Literature Review 


Academic Language/Vocabulary 


Academic language is a well-researched enterprise, and a large family of terms surrounds its 
definition and meaning. Cummins (1999 & 2008) attempted one of the very first distinctions 
between what he terms Basic Interpersonal Communicative Skills (BISC), and Cognitive 
Academic Language Proficiency (CALP). Scarcella (2003: 9) defines it as “a variety of 
English used in professional books and characterized by the specific linguistic features 
associated with academic disciplines.” Within the scope of this research, the most 
comprehensive definition of academic language is presented by Bailey (2007:10) “knowing 
and being able to use general and content-specific vocabulary, complex grammatical 
structures, and multifarious language functions and discourse structures to interact about a 
topic or impart information to others.” General academic vocabulary is variously labeled as 
sub-technical words, specialized non-technical lexis, and common core vocabulary. Coxhead 
(2000:218) defines them as “lexical items that occur frequently and uniformly across a wide 
range of academic material.” Townsend (2009:242) defines them as “words which are used 


9 


across content areas.” Their salience in academic discourse suggests their immediate 
importance in academic language instructions, and their pedagogical potential to augmenting 


the lexical competence of academic writing learners. 


The definition of academic vocabulary begs the question of whether academic writing 
pedagogy should focus on the lexical skills/features salient across different disciplines, or the 
focus be limited to vocabulary needed in distinct disciplines. Hyland (2006) puts forward 
three main reasons why English for Academic Purposes (EAP) instructions should be focused 


on general academic vocabulary: 


1- Language teachers lack the expertise and the level of specialist content control required to 
teach domain-specific language, which could result in misleading students while 
attempting to teach what is essentially not linguistic. 

2- In the process of developing the academic linguistic competence, EAP learners are not 
equipped to engage with the discipline-specific language tasks. They rather require 
preparatory classes that prepare them for general academic English first. 

3- EAP should focus on “the common core” since a finite set of linguistic skills comprise 
almost all academic textbooks, and no significant lexico-grammatical variation exists to 


justify subject-specific academic language instructions. 


Formulaicity of Academic Language 


It is remarkable to reflect on how ‘unoriginal’ the process of language formulation is. 
Recently, several studies have used corpus data to prove how recurrent prefabricated 
formulaic sequences are used in language production. Altenberg (1998), to name one 
example, estimates that 80% of words used in the London-Lund Corpus are parts of recurrent 
lexical combinations. Oakey (2002:112) cites: “a minority of spoken clauses are entirely 
novel creations in the sense that the combination of lexical items is new to the speaker.” 
Phraseology, lexical bundles, word clusters, or more comprehensively, formulaic language 
proliferate academic writing with equal measures. Simpson-vlach and Ellis (2010) maintain 
that large stretches of academic language comprise collocation streams flowing into each 
other, which complies with Sinclair’s Idiom Principle according to which “a language user 
has available to him a large number of semi-preconstructed phrases that constitute single 
choices, even though they might appear to be analyzable into segments” (P. 110). Hyland and 
Jiang (2018) finally maintain that academic writing comprises an inventory of prefabricated 


phrases that is larger in size than that of news and fiction writing genres “with over 450 
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different four-word clusters occurring more than 10 times in one million words” (P. 385). 
Accordingly, the dense formal lexis of academic language comprises “core” conventionalized 
components of vocabulary and formulaic language that could be prioritized and focused on in 


academic writing vocabulary instructions. 


University Students’ Lack of Academic Lexical Competence 


The severity of lack of linguistic competence displayed by university students, 
especially in academic language processing and production variates, yet almost always 
remains perceivable and problematic. Biber (2006) asserts that university students have to 
make major adjustments to handle a range of obstacles and difficulties many of which 
involve using language in new ways, more particularly, developing the ability to understand 
and produce academic discourse. Wong Fillmore and Snow (2000) as cited in Scarcella 
(2003:3) illustrate the magnitude of the problem in California State University reporting “the 
failure rate of the English placement test across the 22 campuses in 1998 was 47%; at one 
campus, it was 85%.” On a less drastic level, Wood (2015) refers to the missing 
“naturalness” in students’ academic writing, which is caused by deviation from the 
conventionalized norms of academic language production. This level of linguistic 
incompetence manifests itself in the case of international students studying in native-speaking 
educational institutes. The enormity of the problem is truly perceived if looked at in non- 
native educational institutes, especially in the Middle East. Alfadda (2012) and Alrabai 
(2016) identified a number of difficulties characterizing Saudi university students’ academic 
writing. While there seem to be several reasons for Saudi students’ lack of proficient 
academic writing, including the profound influence of Arabic language for cultural and 
religious reasons; teacher behavior in class and problems with the Saudi educational system, 


curriculum design and teaching methods seem to be the most difficult conundrums facing 
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both teachers and students. Alrabai (2016) maintains that technology is scarcely used in EFL 
contexts; even technologies as basic as computers, projectors, and recorders seem to be 
outlandish concepts in Saudi educational institutes. Results from Alqahtani (2011) clearly 
indicate writing as the most difficult of the four language skills for Saudi university students. 
In addition to the overarching difficulty with writing, Al-Khairy (2013) cites lack of 
appropriate vocabulary and limited vocabulary size as the most acute problem faced by 
university level Saudi students studying locally and internationally. As a possibly effective 
solution to Saudi students’ lack of academic lexical competence, and in compliance with the 
scope of the current research, the definition, pedagogical theories, and instructional efficacy 
of Computer Assisted Language Learning (CALL) in teaching general academic lexis shall 


be reviewed in details in the following sections. 


Computer-assisted Language Learning: Definition and Overview 


A definition that encompasses the broad applicability of Computer-assisted Language 
Learning (CALL) is “any process in which a learner uses a computer and, as a result, 
improves his or her language” (Beaty 2010:7). While sounding rather loose, the definition 
represents the amorphous nature of CALL and the technology that derives it since it is 
deliverable through a myriad of software and hardware both inside and outside the language 
classroom. Modern CALL has been shaped not only by technology advancement, and 
improved individual digital literacies, but also by theories and pedagogical trends of second 
language acquisition (SLA). While the former factor is progressive and linear in nature, the 
latter is disorganized, and in some cases self-contradicting, which predicts, looking in 
retrospect, how complex the history of computer-language relationship is (Davies, Otto & 


Riischoff 2013). 
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Healy (2016) maintains that, as technology advances, different acronyms on the periphery of 
computers and language dichotomy until CALL. CALL suggested by Davies and Higgins in 
1982, and became the preferred acronym. Mobile-assisted language learning (MALL) 
emerged as a subdivision of CALL. It was used to emphasize learners’ communication over 
the Internet and smart mobile phone application in language pedagogy. Healy (2016:53) cites 
three operational roles/stages of computers in language learning: “behavioristic, 
communicative and integrative.” In the first stage, CALL is based on programs that drill and 
facilitate practice, in the second, the focus is on computer as educational tools, and the third 


is the post-internet era characterized by full integration of computers in classroom practice. 


Using computers in language pedagogy must be grounded on a solid theoretical base, and 
while there are scarce language acquisition theories that are native to CALL Hubbard (2008), 
Hubbard and Levy (2016) maintain that CALL instructions are based on three main second 
language acquisition (SLA) theories namely, The Interaction Account, The Sociocultural 


Theory, and Constructivism. 


Empirical Pedagogical Value of CALL 


On a general note, CALL has been cited as a major contributor to SLA in all four subskills. 
Reading is taught using text-based glosses, online dictionaries, and concordancing tools, and 
using more socio-culturally informed technologies including chat and email (Liaw & English 
2017). Writing is taught using three technology categories, the first is Web 2.0 applications 
for content contribution including wikis and Google docs, the second is automated writing 
evaluation systems, and the third is corpus-based tools such as Corpus of contemporary 
American English (COCA) and Michigan Corpus of Upper-level Student Papers (MICUSP) 
(Li, Dursun & Hegelheimer 2017). Listening teaching and learning is enhanced using several 


digital devices (computers and media players), and Web 2.0 applications such as YouTube 
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and English Central to maximize exposure to authentic native language (Hubbard 2017). 
Finally, speaking is taught and learned using various synchronous and asynchronous CMC 
activities, tutorial, and CALL activities (Blake 2017). On a more specific note, the efficacy of 
CALL in SLA has been empirically proven in modern literature within and beyond the 


boundaries of the four subskills as will be illustrated in the subsequent section. 


CALL seems to pedagogically accommodate diverse learner factors such learning styles, 
gender and age. Lee, Yeung and Ip (2016) conducted a study on a large number (401) of 
students aged 17-36, with 140 male participants and 261 female participants. They collected 
data using a questionnaire designed to examine learning styles, motivation, autonomy and 
computer use. Their results indicated that neither gender nor age has any significant effect on 
learners’ competence to respond to CALL, and that learners with visual and kinesthetic 
learning styles are more readily equipped to use CALL tactile learning style. These findings 
are generally acceptable since no comparison with made with younger participants, e.g. 
elementary students, where competence level could have variated as indicated by (Chiu 


2013). 


In a similar vein, Parmaxi and Zaphiris (2016) synthesized a sizable body of research 
pertinent to the pedagogical value of Web 2.0-based CALL tools in L2 learning contexts, a 
total of 41 studies. Authors investigated a variety of Web 2.0 technologies including web- 
based corpus, wikis, blogs, Facebook, and Google Documents. They concluded that these 
technologies support a range of linguistic and metalinguistic skills including but not limited 
to writing (15/41 studies), interaction and engagement (5/41 studies), intercultural awareness 
(4/41 studies), speaking (3/41 studies) and autonomous learning (3/41 studies). Investigating 


such recent body of literature highlights the ubiquitous presence of CALL technologies in L2 


14 


teaching, and the growing body of scholarly activity that proves the efficacy of CALL 


beyond the mere teaching of the four subskills. 


Investigating the efficacy of CALL in teaching English for Specific Purposes (ESP) can 
illuminate CALL’s affordances in teaching English for Academic Purposes (EAP), which is 
the purpose of the present study. Dashtestani and Stojkovi¢é (2015) reviewed 55 research 
studies on using CALL technologies in ESP instructions namely, course management 
systems (CMS), corpus, electronic academic dictionaries, grammar checkers, plagiarism 
detection software, virtual reality, wikis, blogs and social networking, and mobile devices 
and tablets. They concluded that there is clear empirical evidence for the efficacy of CMS, 
corpora and wikis and hand-held devices in ESP instructions. Based on the number of 
researches reviewed in this study, such findings suggest possible high efficacy for CALL in 


teaching academic vocabulary and formulaic language. 


CALL has also been reported as an effective medium for reading instructions. Two recent 
studies conducted in different EFL contexts and countries namely Saudi Arabia and Iran 
(Alseghayer 2016, Eftekhari 2018) argue for the value of CALL in assisting comprehension 
and promoting reading abilities. The former was teacher-focused where a representative 
sample of 39 male and 31 female experienced college-level reading teachers were surveyed 
with regards to their perceptions about the importance of CALL in L2 reading. Instruments of 
reliability and validity were applied to the questionnaire used, and its results were statistically 
analyzed. It was concluded that on the attitude favorability scale, the surveyed sample had a 
positive attitude about CALL for reading. The latter was a longitudinal study that 
investigated the efficacy of reading via computer versus pen and paper in an EFL context in 
terms of comprehension, recollection and retention of argumentative text. To that end, 120 


undergraduate students were received software-based instructions on argument structures for 


15 


a whole university term, and it was concluded that argument reading via computer has 
significantly supported subsequent comprehension and retention of study material. The 
reason for this positive correlation is argued to be due to the high usability of the software 
that facilitated the construction of arguments faster than pen-and-paper method; hence 


allowing more time for assimilation and comprehension of arguments. 


The pedagogical efficacy of CALL extends itself into teaching particular grammatical 
components. Kilickaya (2013) compared the statistical significance of three different types of 
instructions of teaching adverbial clauses; computer-based instruction, teacher-driven 
instructions, and teach-driven supported by computer based instructions. Pre- and posttest 
results for the 50 EFL learners who participated in the study were compared and it was 
concluded that learning gains for computer-based instructions were higher than traditional 
instructions, and the combined teacher-computer instructions are more effective than 
exclusive computer instructions. The number of participants used in the study qualify for 
possible generalization of results, and promises similar success especially if the taught 
grammar is less complicated. It must be noted here that the study employed explicit rather 
implicit grammar instructions. No clear evidence of incidental grammatical gains could be 
concluded yet this conclusion does not negate the findings reached by Kuppens (2010) that 


incidental grammar acquisition is possible during exposure to media. 


Finally, Grgurovic, Chappelle, and Shelley (2013) present a compelling argument for the 
effectiveness of CALL in their meta-analysis. The authors surveyed the results of 200 
empirical CALL studies conducted between 1970 and 2006 and synthesized the results of 37 
studies that met the inclusion criteria. To decide whether CALL interventions had an effect 
on learning, effect size of several CALL aspects including instructional conditions (duration 


of the intervention, type of technology used, and the degree to which technology was 


16 


integrated), learners characteristics, and research design (assignments required, number of 
participants, and educational settings). They concluded that when comparing between CALL 
and non-CALL groups, within rigorous research design conditions, CALL groups out 
performed non-CALL groups by a small yet positive and statistically significant mean effect 
size of 0.257. In pre- post-test designs, CALL groups were also seen to improve with effect 


sizes that were small yet positive and statistically significant. 


CALL Issues and Limitations 


Despite the theoretical soundness, and the acclaimed pedagogical efficacy of CALL, it has 
received criticism on several accounts. To start with, Wang and Heffernan (2010) commented 
on a number of ethical concerns where CALL could violate personal privacy by 
automatically recording learning history and learning results through cookies, and allowing a 
possibility of divulging what is considered private information including homework, online 
collaborative projects and grades. Mahdy (2013) maintain that CALL has not, and is not 
projected to be “normalized” in class since there are still teachers who believe that computers 
are outside the teacher pedagogical knowledge, and may negatively affect the language 
classroom. He highlights three main concerns for computer transparency and normalization 
in language classrooms. First, personal issues pertinent to teachers and students manifested in 
the lack of time and resources tend to prohibit effective CALL activities in class. Second, 
pedagogical issues, since a large section of EFL courses are still being taught in traditional 
ways, and the majority of textbooks do not readily allow the integration of CALL. Third, 
socio-cultural issues, especially in conservative communities including the Arab world, 
where there are “cultural impediments” (P. 194) that discourage teachers from integrating 
CALL in teaching in the sense that they may contain ethically inappropriate material. Finally, 


Yasmin (2018) emphasizes that the major limitation is that CALL could be liable to lack of 
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teacher training on the use of multimedia in classroom, domination of written communication 
in instructions, assignments and exams, and teacher centered classroom structure that does 
not allow an adequate level of learner autonomy or technology exploration. It is important to 
note, however, that these limitations are not intrinsic to CALL per se, but they are more 
recognized in the way learners and teachers interact with it. One last major issue with CALL 
is that its pedagogical theory tends to have a rather peripheral position in articles that describe 
CALL interventions. CALL researchers tend to be mere consumers of linguistic theories 
developed for other specific purposes. Such a tendency ignores or at least minimizes the 
prominence of technology in the learning language-learning environment, or more 
dangerously customizes the theory itself to accommodate the new technology-driven context 


(Stockwell 2016). 


On a more specific note that is more pertinent to the present study, integrating CALL 
in EFL Saudi settings has been facing a number of impediments. Integrating CALL in EFL 
settings imply the use of computers to complement language instructions not to supplement it 
so that the learning experience is enhanced with the resources and opportunities made 
available by computers and would not be available otherwise. However, Al-Rasheed et al. 
(2015) maintain that Saudi Ministry of Education and Ministry of Higher Education strictly 
regulate curricula development in all educational institutions in Saudi Arabia, and they tend 
not to motivate or encourage the integration of CALL in language classrooms. Furthermore, 
no visible effort is made to provide teachers with software that enables them to create CALL- 
informed lessons or programs. Al-Kahtani and Al-Haider (2010) delineate several overt and 
covert factors that inhibit the use of CALL in language instructions. They surveyed four 
Saudi Universities and found that female faculty members were not provided with sufficient 
computing facilities and that while financial support was adequately provided to female 


instructors, essential training and institutional and technical support were missing. Another 
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important, yet less impactful factor was lack of interest on the side of teachers to use 
computers in language teaching. Despite the scarcity of technology, and other cultural and 
organizational limitations, Al-Kahtani (2011) and Sagqlian et al. (2013) report positive 
attitudes and readiness on the part of English language instructors to use CALL in EFL 
instructions. In the two cited studies, most faculty members believe that the integration of 


CALL would be beneficial to teaching the four subskills. 


Mobile Assisted Language Learning 


Mobile assisted language learning (MALL) is a research topic within the field of CALL that 
provides solutions to a number of limitations of CALL including but not limited to over 
emphasis on delivery method, dense workload, and inadequate teacher training. With its 
portability and the built-in capacity to display images, videos, and texts, the potential of 
mobile phones in language education can be hardly doubted. Nevertheless, the physical 
characteristics of the screen and input, and the overwhelming skeptical attitude on the part of 
learners can represent unresolved problems. According to Stockwell (2016), research on 
(MALL) can be broadly categorized into research under controlled conditions and research 
outside the classroom with the majority of research focusing on vocabulary and listening 
teaching. He puts forward a number of principles that must be considered while designing 
MALL instructions, chief among which are limiting multitasking, keeping MALL activities 
short, and using texting with boundaries. Miangah and Nezarat (2012) cite a number of 
properties that characterize MALL through various devices with special emphasis on mobile 
phones including portability, social interactivity, context sensitivity, and individuality. Such 
affordances of mobile phone devices allows numerous pedagogical opportunities in EFL 
contexts. The general research trends and pedagogical value of MALL are explained 


hereunder. 
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Empirical Pedagogical Value of MALL 


In their recent review of 78 MALL-related articles, Elaish et al. (2017) maintain that MALL 
is capable of supporting almost all areas of SLA namely, vocabulary, speaking and listening, 
comprehension, and writing with proven advantages including connectivity, spontaneity, and 
collaboration. They highlighted the ubiquity of MALL research in EFL contexts. However, 


the review did not tackle the effect size of MALL interventions in the selected articles. 


In another review of MALL publications, Hwang and Fu (2018) surveyed the research design 
and applications of mobile language learning and found that the number of studies using 
mobile education has increased significantly in the last five years. It was also observed that 
the scholarly body of research is moving from teaching individual language skills to 
delivering multiple language skills using authentic materials. They also observed that MALL 
brought high effectiveness in terms of speaking, vocabulary, writing and pronunciation in the 


majority of reviewed articles. 


Empirically, Rosell-Aguilar (2018) studied learner autonomy afforded by MALL, the case of 
a particular language learning mobile application, busuu. The article cites results from a large 
sample of 4095 participants whose data were collected by means of online survey. Findings 
suggest that the applications helped them improve their language knowledge, with 
vocabulary being the main area of development. It is clear that the large population 
employed, and data collection method used in this study qualifies generalization of findings 


into similar pedagogical settings. 


Another case study of L2 learning mobile application Duolingo by Loewen et al. (2019) cites 
findings of a semester-long intervention for nine participants learning Turkish. Participants 
showed improvement in L2 measures including the four subskills and learners’ lexico- 


grammar with positive moderate correlation between learning gains and time spent using 
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Duolingo. The study, while present positive results for MALL, its results can be generalized 


cautiously since the sample size was limited to eight participants. 


Finally, in their meta-analysis, Sung, Chang and Yang (2015) synthesized the results of 44 
peer-reviewed journal articles on the effect of language learning using mobile devices, and 
reported a meaningful improvement in all taught skills using MALL with a medium effect 
size of 0.55 with 95% confidence interval of 0.387—0.705. These findings suggest that 70.7% 


of learners who used mobile devices performed significantly better than those who did not. 


Human Cognition and Vocabulary Learning 


The seminal work of Ellis (1995) and the more recent of Allum (2004) emphasize not only 
what could be done with computers in vocabulary teaching, but also the importance of human 
cognition during vocabulary instruction. What is in a word to learn is an important 
preliminary question that precedes assessing the effectiveness of computer-assisted 
vocabulary instruction (CAVI). For a word to be stored in the human mental lexicon, a new 
orthographic pattern must be recognized first, and then semantic and phonological links must 
be made to facilitate future recollection of the new lexical item. Additionally, syntactic 
properties of the item must be learned along with its place in the lexico-grammatical 
environment, and its collocational or primal relations with other words. Accordingly, an 
effective vocabulary-learning framework should comprise frequent exposures to target 
vocabulary in an explicit mode of instruction while involving definitional and contextual 
lexico-grammatical information, a major pedagogical implication for CALL. Nakata (2008) 
persuasively argues that using multimedia, for instance, in vocabulary instructions reinforces 
lexical retention because they allow diverse access routes to the learned lexical item allowing 


a deeper memory trace for the learned vocabulary. 
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Further implications from cognitive psychology cited by Nakata (2008) imply that 
relying on context alone in vocabulary acquisition is not sufficient. Complementary activities 
that involve the explicit use of word lists are more inductive of learning and more efficient. In 
that context, academic vocabulary word lists, an immediate outcome of corpus linguistics 
research, become crucial since they “bring some order to what otherwise, would be a 
vocabulary chaos” (Gardner & Davies 2014:310). Nakata (2008) emphasizes that the use of 
word lists as the base of vocabulary instructions should be founded on spaced learning, and 
must allow several opportunities for vocabulary recollection because L2 vocabulary retrieval 
reinforces memory retrieval routes. Similarly, Mayer and Moreno (2003) propose that 
multimedia learning allows more effective learning based on having multiple input channels 
that allow auditory and visual dual coding, which reduces the cognitive load (the quantity and 
quality of information required to be processed) at the point of vocabulary instruction, which 


is another clear affordance of CALL. 


CALL Efficacy for Vocabulary Teaching 


Available literature empirically shows CALL’s efficiency in vocabulary instructions. Tozcu 
and Coady (2004) found that learners who used CALL tutorials had higher gains in acquiring 
highly frequent vocabulary than the control group, and had a better recollection time. 
Kilickaya and Krajka (2010) compared the performance of an experimental group that used 
an online tool for vocabulary instruction and that of a control group that used paper and 
cards, and found that the former cohort outperformed the latter, and had showed longer 
vocabulary retention in delayed post-test. Similarly, Shoaei and Alavi (2016) report a 
significant impact for computer-assisted vocabulary learning using multimedia application in 
terms of vocabulary recollection and retention. A synthesis of the research available on 


computer assisted vocabulary acquisition is presented in Chiu (2013) meta-analysis, which 
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shows a positive medium effect size (d = 0.754, p =0.000) of CALL activities. The meta- 
analysis that scrutinized the empirical research of 16 L2 vocabulary CALL instructions also 
showed important findings related to learners’ educational level and duration of instructions 
where the effect size of CALL vocabulary instructions for interventions that lasted less than a 
month is higher than in those of more than one-month duration. It also shows that high school 
or university students benefit more from CALL activities compared to younger learners. It is 
evident from the articles reviewed that academic lexico-grammar teaching using CALL is a 


territory with potential for further exploration. 


MALL Efficacy for Vocabulary Teaching 


As far as vocabulary teaching is concerned, MALL has three pedagogical affordances: 1) 
allows incremental vocabulary input over extended periods of time, which decreases 
cognitive load. 2) Allows repeated exposure to target vocabulary through either text 
messaging or smart phone applications, which accelerates the internalization of target 
vocabulary. 3) Allows opportunities of explicit and incidental vocabulary learning (Li et al. 
2017). Using short text messaging has been a dominant medium of instructions in recent 
empirical research. Lu (2008) taught 14 words using text messaging to an experimental group 
during their commuting time, post-test result showed better vocabulary gains compared to 
control group. Gurocak (2016) complimented an academic vocabulary course with by 
sending target vocabulary items from Coxhead’s academic wordlist via text messages to 
learners outside classroom over a period of eight weeks and reported high vocabulary gains 
for the experimental group in terms of semantic appropriateness and grammatical accuracy, 
and positive learners’ attitude towards MALL vocabulary instruction. MALL’s affordance to 
enhancing academic vocabulary was also reported in Li, Cummins and Deng (2017) who 


used frequent structured exposure via text messaging to highly salient vocabulary items from 
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ce 


Cobb’s VocabularyProfilers academic wordlist and concluded that text messaging is “a 
malleable and gratifying means of vocabulary instructions” (P. 13). The cited results suggest 
that MALL, particularly text messaging can prove quite efficient in teaching academic 


vocabulary. 


Using CALL in academic writing, then, does not necessarily require textual analysis using 
concordancing tools (data-driven learning) exemplified in the work of Kaur and Hegelheimer 
(2007), and Greaves and Warren (2007). A selection of academic vocabulary derived from 
Oxford Phrasal Academic Lexicon can be explicitly taught using Web 2.0 platforms such as 
Quizlet and Kahoot. It can then be reiterated and practiced via mobile phone texting 
applications, such as Whatsapp. Employing academic pedagogical wordlists to inform a 
computer assisted vocabulary-learning intervention that is paired with out-of-class practice 
made possible via smart phones’ applications promise an interdependent, thorough and 
potentially highly effective application of CALL for English for Academic purposes 


vocabulary instructions. 
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Chapter three: Methodology 


3.1 Research Intent 


The reviewed literature unveiled the appreciable significance of general academic vocabulary 
along with the several pedagogical affordances for computer and mobile assisted instructions 
in relation to vocabulary acquisition. It also showed the value of input frequency and the 
cardinal importance of vocabulary recollection in reinforcing memory retrieval routes, and 
emphasized the fact that the maximum effect size for vocabulary acquisition intervention is 
produced when the intervention is rather short, that is one month in duration or less. Based on 
these contributions of previous research, the intent of this study is to design and appraise a 
general academic vocabulary instruction intervention geared towards improving the lexico- 
grammar of Saudi university students’ written production. Target vocabulary is directly 
derived from Oxford Phrasal Academic Lexicon (OPAL). CALL is used as the main medium 
of instructions, in-class activities and exercises, and MALL is employed to compliment the 
intervention’s activities outside class, and enhance target vocabulary’s entrenchment and 


priming in student’s academic repertoire. 


3.2 Research Context 


The reality of English as Foreign Language (EFL) teaching in Saudi higher education is 
exactly as unwelcoming as described in the literature, perhaps more imperfect, let alone 
English for Academic Purposes (EAP). Two major predicaments characterized the context of 
the present research, namely students’ acquaintance with foreign language, and institutional 
preparedness to use technology in EFL teaching. On the one hand, Arabic LI, is used 
extensively to teach L2. The majority of students and teachers adamantly hold the belief that 
Arabic must be used in English language classes especially for the purposes of explaining 


grammar, and introducing new vocabulary. Teachers claim that using L1 in L classes is time 
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efficient and more productive in terms of new language processing. That claim is falsified by 
Alarabi (2016) who maintains that the real reason behind using L1 in EFL classes is teachers’ 
lack of linguistic competence, and their desire to take instructional alternate routes. The 
immediate result of that behavior is students’ inability to use target language in 
communication, which undermines their overall communicative competence. The concepts of 
academic vocabulary and academic language sounded entirely foreign to students who were 
immediately struck by the “novelty” of the idea. Accordingly, it is not a matter of fluency 
deficiency, but rather a deep-rooted linguistic incompetence. On the other hand, the 
university at which the experiment was conducted lacked essential technological 
infrastructure. Access to the computer lab was not possible; accordingly, students used their 
personal computers. Available delivery hardware was limited to a portable projector and a 
white board. Implementing a CALL experiment in this educational setting was a daunting 


challenge. 


Prior to commencing the experiment, research ethical obligation form was completed and 
approved by the university, where the procedure of recruiting participants, getting their 
informed consent of involvement, handling participants’ confidential information, and 
avoiding coercion were clarified, acknowledged and approved. Subsequently, necessary 
approvals were attained from Qassim University is Saudi Arabia where the Deanship of 
Educational Services (preparatory year program) officially corresponded with the University 
of De Montfort, UK and provided official consent for the researcher to conduct research and 


collect data. 
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3.3 Design 


The reported study in the following sections is quantitative and fully experimental in nature 
with an experimental group and a control group that is intended to measure the exact efficacy 
of each component of the suggested delivery instruments on the written production of the 
selected populations using a pretest-post-test data collection format. The proposed study 
comprises 8 in-class two-hour sessions delivered over a period of one month, and 40 
structured questions intended to reinforce the acquisition of target vocabulary outside 


classroom. 


3.3.1 Delivery Instruments 


1) Oxford Phrasal Academic Lexicon (OPAL). 


Core sub-technical academic vocabulary that are not domain specific account for a 
considerable coverage of the words used in academic discourse because the method 
employed in selecting them considers two major factors, words’ range and frequency. Range 
refers to the selected words’ occurrence across various disciplines verified by using a large 
corpus that comprises millions of words from different academic domains. Frequency refers 
to the number of times the words’ families occurs in the academic corpus (Coxhead 2000). 
Teaching words from pedagogical academic word lists guarantees principled vocabulary 
selection that is not based on teachers’ intuition, and ensures that the words chosen are 
frequently employed in various academic texts. Academic formulaic language should be 
pedagogically viewed with equal measures of significance. Frequent lexical clusters can be 
thought of as extended collocations that systematically appear in academic texts at a 
frequency that cannot be predicted by chance, and contribute both meaning and cohesion to 
academic written production (Hyland 2012). While there were several available word and 


formula lists (Gardner & Davies 2014, Coxhead 2000, Simpson-vlach & Ellis 2010, Morley 
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2017), OPAL was selected for this experiment for a number of reasons. First, it comprises 
both academic words and formulas in a web-based user-friendly interface equipped with 
search bar that returns immediate results with examples. Second, it presents word lists and 
formula lists in the formats of sub-lists and functions respectively. Vocabulary sub-lists are 
categorized based on frequency where sub-list 1 has the most frequent, hence the easiest 
academic words while sub-list 10 has the least frequent; hence the most difficult. This feature 
allows easy selection of words that correspond to the language level of learners. Formula 
function categorization allows accessibility and navigation through the functions and their 
linguistic representations. Three, it integrates a word pronunciation facility which can allow 
better lexical retention through diverse access routes, and auditory and visual dual coding of 
learned words. Finally, OPAL comprises the most word academic word list, available as of 
January 2019, and it is directly linked to Oxford Learner’s Dictionary of Academic English, 
which allows a wide range of examples that promise faster entrenchment and collocational 
priming of target vocabulary. After choosing OPAL, and based on the level of students 
determined by the university’s placement test, subsets 1, 2, 3, and 7 were selected for the 
vocabulary intervention, every subset comprised 10 root words, a total of 40 words. Subsets 
1, 2, and 3 represented the most frequent, highly salient root words in academic writing. 
Subset 7 represented a slightly less frequent set of words chosen in order to balance the 
difficulty level of the intervention so that it would not be too unchallenging. The intervention 


presented the following root words: 


1- concept, policy, finance, procedure, derive, establish, require, specific, interpret, 
estimate. 

2- effect, participate, consequence, perceive, impact, maintain, aspect, evaluate, obtain. 

3- imply, sufficient, outcome, constraint, proportional, justification, validity, illustrate, 


component. 


28 


7- advocate, ultimate, foundation, empirically, prioritize, eliminate, infer, phenomenon, 


isolation. 


In a similar vein, since hedging is, perhaps, the most prominent feature of academic 
language, a number of formulaic structures that denote cautious language were also selected 
from OPAL including, may, seem to be, possibly, thought to, and tend to, and introduced as 
part of the intervention. Formulaic language of other academic discourse functions such as 
exemplification, reference to evidence or previous research, comparison, addition, and 


concluding were also introduced but with less emphasis. 


2) Reading Texts 


In an attempt to construct academic/scholarly voice/language within the indisposed, 
conversational and generally faulty linguistic repertoire of the Saudi students, form-meaning 
mapping (Ellis et al. 2015) had to be accounted for. Words were introduced as learnable pairs 
of forms and their associated semantic function. Accordingly, Target vocabulary and 
formulaic language were contextualized in four short essays; each one was tailored to 
accommodate ten root words with various inflections and three to five formulaic constructs. 
Essays were color coded so that target lexical items (vocabulary or formulaic) and their 


collocational associations were distinguishable from the rest of the text. 


3) Quizlet 


Vocabulary acquisition research refers to two distinct modes of vocabulary learning namely, 
incidental and intentional learning. While incidental learning of vocabulary can lead to 
tangible gains in vocabulary acquisition, explicit focus on vocabulary learning leads to faster 
and more significant gains in terms of vocabulary retention (Schmitt 2008). Laufer (2005) 
reports 33-86% more words learned using explicit instruction compared to using incidental 


learning. Accordingly, research points at the direction of explicit vocabulary teaching in EFL 
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classrooms as a more effective method of vocabulary teaching. It also suggests that 
flashcards, digital or conventional, represent an effective explicit vocabulary-teaching tool, 
that allows students to retain more words in a shorter time (Nation 2001, Fitzpatrick et al. 
2008, Sitompul 2013, and Hung 2015). As a free digital flashcards platform, Quizlet provides 
customizable card templates where target vocabulary, examples, and pop-quizzes can be 
presented and maneuvered. Being a web-based application, accessing it is quite easy from 
personal computers or tablets, and integrating word definitions for review is made possible 


through the editable user-friendly interface. 


4) Text Messages 


Several quantitative and qualitative studies (Lu 2008, Cavus & Ibrahim 2009, Gurocak 2016, 
and Li et al. 2017) proved that using text messaging in EFL learning environment can expand 
learners’ vocabulary knowledge, and can allow acquisition of target vocabulary that is more 
permanent. Previous research also referred to learners’ positive attitude towards vocabulary 
learning using text messages, and reported it as being more resourceful and more motivating 
compared to traditional printed formats. Accordingly, given the language level of 
participants, the relative short duration of the intervention, and the number of lexical items 
targeted, employing MALL was thought of as a cognitive advantage that could enhance the 
retention of newly introduced items. Therefore, a WhatsApp group that included all members 
of the experimental group was created, and intermittent text messages were sent from 
researcher to group members. Messages were intended to allow several opportunities for 


vocabulary recollection because L2 vocabulary retrieval reinforces memory retrieval routes. 


3.3.2 Data Collection Instruments 


1) Pretest-Posttest 
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Essay-writing tasks were used to collect linguistic data from control and experimental groups. 
Participants were instructed to write two 250-word opinion essays within | hour, one prior to 
the beginning of the intervention, and another at its end. Essay topics were modeled after 
IELTS task 2 writing tasks. The essay prompts were selected to engender different responses 
to similar central idea in order to avoid possible discrepancy in the written production’s 


linguistic variables. 


Pretest prompt: Some people think that modern technology is making people more sociable, 


while others think it is making them less sociable. Discuss both views and give your opinion. 


Posttest prompt: ‘Nowadays the way many people interact with each other has changed 
because of technology.’ In what ways has technology affected the types of relationships that 


people make? Has this been a positive or negative development? 


2) Rubric 


A common definition for scoring rubrics is that they are descriptive/qualitative grading 
schemes developed to guide the rating of complex student work (Moskal & Leydens 2000). A 
rubric must have evaluation criteria, quality definition for each criterion at particular levels, 
and a scoring strategy (Dawson 2015). The benefits of using rubrics are many, but the most 
important elements include increasing judgement consistency among raters and across 
students, and providing valid judgements for students’ performance that is unachievable by 
means of traditional writing tests. In the framework of the current study, an analytical rubric 
(versus a holistic rubric) was designed to measure six formalized criteria using specific 
descriptors. Prior to using a rubric for performance evaluation, however, the rubric was 


calibrated in order to answer several questions of scoring reliability and validity. 
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Validity of Scoring 


The American Educational Research Association, the American Psychological Association, 
and the National Council on Measurement in Education (2014) maintain that “validity refers 
to the degree to which evidence and theory support the interpretations of test scores” (P. 11). 
Rubric validation hence is the process of collecting evidence to prove that the inferences 
made about students’ performance and the manner in which the rubric interprets students’ 
performance into quantifiable scores are scientifically sound. The most frequently examined 
evidence to support the validity of a scoring rubric include content-, construct- and criterion- 


related evidence. 


Content-related evidence is concerned with the degree to which students’ responses sample 
the content domain, and refers to rubric’s ability to guide the rater to evaluate students’ 
knowledge of the precise content area being assessed with no distraction from other 
qualitative features that should not influence the interpretation of students’ responses. For 
instance, while developing a rubric for a chemistry-focused content, the teacher could 
unintentionally shift from the intended chemistry content to emphasizing the non-chemistry 
components of the task such as sentence structure or spelling. Accordingly, resultant scores 
would be more of a reflection of students’ grammatical knowledge than their chemistry 


knowledge. 


Construct-related evidence is concerned with the rubric’s ability to measure constructs; 
internal processes of students not just unrepresentative manifestations of these processes. For 
instance, reasoning is an internal process, yet it might only be partially displayed in students’ 
responses. A scoring rubric hence should contain criteria that evaluate both the product (the 


answer), and the construct/process (the explanation). 
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Finally, criterion-related evidence supports the extent to which students’ performance is 
generalizable to other relevant activities. That is to say, the scores made possible by a rubric 
in the task being evaluated is valid if they suggest high performance practically outside the 


classroom. 


Moskal and Leydens (2000) argue that the purpose of the intended assessment 
indicate which one of the three evidence types should be considered. Accordingly, clear 
learning objectives were set prior to developing the rubric used in the present study, 
evaluating criteria were then created, and a reflection on the link between objectives and 
criteria was carried out to examine each type of validity evidence. The framework for rubric 
design is illustrated in table 1 based on validation questions of evidence validity proposed in 


Moskal and Leydens (2000, P.3). 


Learning Objective Evaluation Criterion/ Criterion Validation 
Descriptors 

Acquiring the largest Target Vocabulary Count. | The criterion does not 

number of the 40 target address any extraneous 


: Six to ten academic words 
vocabulary words in content. 


: . ‘i were used. 
different inflectional forms 
It accounts for moderate 


lexical gains given the low 
linguistic level of 

Two or less academic words __| participants. 

were used. 


Three to five academic words 
were used. 


Using the acquired Target Vocabulary Usage. | It accounts for designed and 
vocabulary correctly incidental acquisition of 
(spelling-grammar- collocation and syntactic 
collocation) environment. 


S shows ability to always (0 
to 1 error) use target words 
appropriately in terms of 
spelling, collocation and 
grammatical usage 


S shows ability to use target 
words appropriately in 
context most of the time (2 to 
3 errors), occasionally 
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making mistakes with 
spelling, collocation or 
grammatical usage 


S is consistently unable to 
use target words 
appropriately (more than 3 
errors) in terms of spelling, 
collocation or grammatical 
usage 


Improving the overall 
grammatical competence 


Holistic Sentence Structure 
& Context 


Sentences are attempted with 
strong context that highlights 
S understanding of the 
meaning of the target 
vocabulary 


Sentences are attempted but 
do not contain context to 
support meaning of target 
vocabulary 


No sentences or very few 
sentences were attempted. 
All or most parts not 
completed. Attempted 
sentences do not offer 
context to support meaning 
of target vocabulary 


Addresses intended aspect 
within the overarching 
lexical framework of the 
learning objectives. 


Improving the overall 
academic tone 


Scholarly & Academic 
Tone 


Strongly present 
Moderately present 


Nonexistent 


Use of Cautious Language 


S employs hedging markers 


Accounts for the acquisition 
of academic formulaic 
sequences 
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appropriately in written text 


S employs insufficient 
hedging markers. Claims 
occasionally sound too 
absolute. 


S does not employ any 
hedging markers or use them 
incorrectly. 


Improving the essay Essay Structure Specifically measures the 


structure . . target aspect 
The written production alee 


presents a clear thesis and 
follows correct structure of 
distinct introduction, body 
and conclusion 


The written production 
presents an 
unclear/incomplete thesis 
and/or follows an incomplete 
(one component is missing) 
or unbalanced (parts of the 
essay are not equal in length 
or focus) structure 
(introduction, body, 
conclusion) 


The written production is 
missing thesis and/or 
structure misses two or more 
components (introduction, 
body, conclusion) 


Table 1: Framework for designing the Grading Rubric 


Furthermore, criteria explanations and descriptors’ definitions were provided to the two raters 


who volunteered to grade the pre- and post-tests using the following scoring instructions: 
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1- This assessment is lexically oriented. That is, vocabulary is the main subject of your 
judgement. 

2- Your judgement should not be influenced by the mechanical characteristics of 
students’ writing. 

3- The count of academic vocabulary used based on the provided list is a very important 
metric. So kindly, make sure this is accurately calculated. 

4- Ifthe exact same academic word is used more than once in the same essay, it should 
be counted as one instance of use. 

5- Ifthe same academic root word is used in different inflectional forms, each instance 
should be counted as a new word. 

6- Correctness of lexical items’ usage is critical. Target vocabulary word must be used in 
the correct form and in the correct lexical environment. 

7- Essay structure and theses clarity should be meticulously graded. 


Important definitions: 


1- Hedging markers refer to any word that limits the force of claims made in order to 
give the written production a more academic voice. Examples are (perhaps, likely, 
may, might... etc.) 

2- Scholarly & Academic Tone refers to the formal, impersonal, objective voice that is 
characteristic of academic writing. It avoids the use of personal or emotional 
language, over-sweeping adjectives, and colloquialism. 

3- Essay structure refers to essay’s parts (introduction, body and conclusion). 


Reliability of Scoring 


In terms of reliability, Moskal and Leydens (2000) maintain that an evaluation rubric with 
explicitly clear scoring categories should enable consistent scoring notwithstanding different 
raters or the timing in which rating is completed. In that sense, two reliability metrics should 
be accounted for while designing a rubric, intra-rater reliability that is consistency of a single 
rater over time; and inter-rater reliability that is consistency across raters simultaneously. The 
first metric ensures that no external factors to the purpose of the evaluation are affecting the 
manner a in which a rater scores students’ responses, for instance, a rater can become weary 
as he/she evaluates students’ work, and hence devote less attention to critically analyzing 
responses over time. Accordingly, the same student response may receive different scores 
than he/she would have had earlier during the evaluation process. In their exhaustive review 


article, Jonsson and Svingby (2007) maintain that intra-rater reliability is investigated using 
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Cronbach’s alpha with a reference alpha value that is above .70. In order to determine intra- 
rater reliability score for the two raters participating in the current research, each one was 
asked to use the designed rubric to rate the same short essays produced by 15 students 
studying an English for Academic Purposes (EAP) course in two different instances with an 
interval of two weeks. Each rater’s scores were the analyzed using IBM SPSS 24 statistics 
package to obtain their individual Cronbach’s alpha scores, and their scores were .783 for the 
first rater, and .769 for the second, denoting a generally sufficient level of intra-rater 


consistency. 


Inter-rater reliability refers to the degree of consistent agreement between raters while using a 
rubric for scoring students’ performance. Frequent scoring consensus refers to the fact that 
the rubric’s descriptors are clearly distinguishable and easy to interpret, descriptors are 
attributed scores that are easy to calculate, and descriptors can be thought of as nominal data. 
To quantify inter-rater reliability, Jonsson and Svingby (2007) cite the use of Cohen’s kappa 
to estimate the degree to which rating consensus vary from the rate expected by chance, and 
they maintain that Kappa values between .40 and .75 do represent fair agreement beyond 
chance. To evaluate inter-rater reliability the designed rubric can ensure, raters’ scores of the 
15 EAP students were analyzed using SPSS in the two instances with a two-week interval. 
Kappa scores for the two raters were .684 in the first instance, and .633 in the second 


denoting a fair level of consistency between the two raters beyond sheer chance. 


3.4 Participants 


The experimental group comprised 30 students in a Saudi University aged 18 to 25. Their 
first language was Arabic and their overall English language proficiency ranged between A2 
and B1 on the Common European Framework of Reference’s (CEFR) scale. Their language 


level was determined by the local placement test administered by the university. All but one 
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participant shared the same EFL background of studying English in governmental schools as 
of the secondary level of education (3 to 5 years of EFL instructions) where L1 was always 
used as the language of instruction in L2 classes, and teachers were never allowed to use 
authentic material. One participant studied in an international school. The control group 
comprised 15 participants of identical specifications to the experimental group, yet only four 
students sat for both the pre- and post-tests. None of the participants acknowledged living in 
an English-speaking country at any point of their lives, or studying English for Academic 
Purposes per se. All participants volunteered to participate in the present study, and 
acknowledged in writing non-coercive behavior on the part of the researcher throughout the 


study. 


3.5 Procedure 


Both experimental and control groups were requested to write the pre-test essay in class, then 
the intervention formally initiated by separating the control group cohort and allowing them 
to study from Cambridge’s textbook Unlock 2 following traditional Saudi unplugged teaching 
methods. The experimental group was initially introduced to the essentials of academic 
writing with special emphasis on the following features: the use of impersonal language, the 
meaning of general academic vocabulary, the form and structure of an essay. Consequently, 
the first set of academic vocabulary was introduced using Quizlet’s digital flash cards. 
Contextualized academic vocabulary and formulaic language were then introduced using 
tailored short essay. At this point, very close attention was given to emphasizing the 
meaningful context of each lexical item, and guide students to consciously study how words 
or bundles syntactically behave, what patterns they creates and can be involved in, and how 
to recognize and independently create similar patterns. The intent was to instill the notion of 


linguistic patterns in the minds of participants to facilitate future production. Extensive 
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exercise was then introduced in order to explore target vocabulary in various contexts. After 
leaving class the following model messages was send to the created Whatsapp group: 
- The definition (definition given) describes which of the following words? (multiple 
choices provided) 
- The definition of the word (word given) is: (multiple choices provided) 
- Amore academic variation for the word (word given) is .............. (multiple choices 
provided) 
- The correct word that collocates with (word given) is: (multiple choices provided) 
- To write cautiously, which word of the following can be used? (multiple choices 
provided) 
- Fill in the space with a word that limits the force of claims made. (multiple choices 


provided) 
The same routine was repeated throughout the eight two-hour sessions, each time, new 
lexical items are introduced in class, followed by in-class practice in groups, then out of class 
word-recollection exercise using mobile phone. At the end of week four of the intervention, 
both experimental group and control groups were asked to sit for the post-test task under the 
exact same conditions of the pre-test. Two experienced raters volunteered to score the written 
production of both groups using the designed rubric. Statistical analysis was conducted using 


IBM SPSS statistics package version 24 for Windows. 
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Chapter four: Results and Discussion 


4.1 Data Analysis 


To conduct thorough analysis of data collected using the designed rubric, the pre- and post- 
test results provided by each rater were compared separately for both experimental and 
control groups. The dataset comprised seven paired variables per rater representing rubric’s 
six criteria namely, target vocabulary count, target vocabulary usage, holistic sentence 
structure & context, use of cautious language, scholarly & academic tone, and essay 
structure, and the total score given for the complete written production. The rational for this 
level of analysis is to reveal the specific areas that were influenced by the intervention if at 
all. Consequently, two levels of analysis were used; the first is the f-test to evaluate the 
validity of the null hypothesis by comparing mean ratings of the pre- and posttests; hence 
proving/disproving statistical significance p of the intervention’s results. The second is effect 
size represented by Cohen’s d, which describes the mean difference between or within 
groups. The latter measure cannot be calculated using SPSS, accordingly it could be done 
either manually or using a specially designed calculator. In the present study, the calculator 
developed by David Wilson was used. It is freely downloadable from the following URL: 


http://mason.gmu.edu/~dwilsonb/downloads/ES_Calculator.xls. The rationale for using effect 


size in the subsequent analysis is that according to Plonsky (2015), the null hypothesis 
significance testing (NUST) alone can be unreliable, and can be easily influenced by the 
sample size, and it does not provide information about the extent of the relationship being 
examined. Conversely, effect size provides an accurate estimate of the actual strength of the 
relationship being examined. Finally, unlike p values, d values are scale free, which allows 


cross-study comparisons. 
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To conduct the analysis, all variables were coded with the prefixes prl/2, pol/2 to 
stand for pre- and posttest results for the first and second raters. A standard f-test was carried 
out using IBM SPSS statistics version 24 for windows to compare the results of pre- and 
posttests, which represent the performance of both control and experimental groups. 
Consequently, the effect size d was calculated separately using David Wilson calculator. 
Results were then compared to findings of previous results in literature review, which will be 


thoroughly discussed in the following section. 


4.2 Results 


Prior to conducting the experiment, the following research questions were put forward: 


1. Can Computer Assisted Language Learning (CALL) improve pre-intermediate Saudi 
university students’ academic vocabulary in terms of quantity and quality? 

2. Can CALL improve the grammatical and/or structural features of pre-intermediate Saudi 
university students’ academic writing? 

3. Can CALL improve the overall academic voice in the written production of pre- 


intermediate Saudi university students? 


Accordingly, the researcher proposed the following hypotheses: 


1. Using digital flash cards afforded by the online platform Quizlet, and the academic 
vocabulary and phraseology provided by the web-based Oxford Phrasal Academic 
Lexicon (OPAL) will enhance the experimental group’s academic vocabulary, and will 
afford more reliable vocabulary internalization and operationalization measurable by 
analyzing the results of the posttest. 

2. The combined pedagogical forces of computer and mobile assisted language learning will 


enable a more effective retention of academic vocabulary and phraseology. 
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3. The academic voice installed in the lexico-grammar of academic language will 


consequently be enhanced through the use of impersonal and cautious language. 


4.2.1 Results for research question one 


Investigating the first two variables, namely target vocabulary count and usage shows that the 
p-value for the variables measured through the pre- and posttests for the experimental group 
by the two raters is .000, which is less than the level of significance benchmarked at .05. The 
null hypothesis is therefore rejected denoting a significant statistical difference between the 


mean scores of the experimental group’s work before and after the intervention. 


Paired Differences p-value 
Mean Std. 95% Confidence 
Deviatio Interval of the 
n Difference 
Lower | Uppe 
r 
Pair priTarget V - 1.0726 - - 000 
1 Count - polTarget 1.233 1.6339 | .8328 
VCount 3 
Pair priTarget VUsage - 1.5071 - - 000 
2 - pol Target 1.266 1.8294 | .7039 
VUsage i 


Table 2: Paired Test for Vocabulary-related Variables for two Raters 


To calculate the effect size d for the paired results, the difference between means of pre- and 


posttest scores is divided by the difference between their standard deviation using the simple 


M1-M2 


formula d erie Table 3 shows means and standard deviations of vocabulary-related 


variables. 
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Mean N Std. Deviation 

Pair 1 | prlTarget V Count 133 30 3457 
polTarget VCount 1.367 30 9994 

Pair 2 | prlTarget VUsage 233 30 MS pe 
polTarget VUsage 1.500 30 1.3834 


Table 3: Vocabulary-related variables’ Means and Standard Deviations 


Accordingly, based on the assessment of the two raters, the effect sizes for vocabulary count 
and correct usage are 1.6 and 1.14 respectively. To understand the meaning of these figures, 


Plonsky (2015) cites a general scale for interpreting d values in L2 research (Table 4). 


Effect size Small Medium Large 
d-value between 0.40 0.70 1.00 
groups 
d-value within 0.60 1.00 1.40 
groups 


Table 4: Benchmarks for interpreting d-value in L2 Research 


In the light of the benchmarks cited, d= 1.6 denotes a large effect size for the intervention in 
terms of the number of academic vocabulary used by the experimental group, while d= 1.14 
denotes a medium effect size for the intervention in terms of academic vocabulary use in the 


correct form and the correct lexical environment. 


4.2.2 Results for research question two 


Results for research question two can be obtained by scrutinizing variables three and six that 
assess the correctness of holistic sentence structures of the treatment group’s written 
production, and the form correctness of written essays structure. As illustrated in table 5, the 
p-value of the paired test for variable three for rater one is slightly different from that of rater 


two, yet both denote statistically significant difference between the results of the pre- and 
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posttests, where p= .002 and .001 respectively. However, the results show a small effect size 
if compared to L2 research benchmarks, where d= .59 for rater one and .77 for rater two, 
denoting that students seem to have improve in terms of the overall grammatical accuracy of 


their written production, yet not quite dramatically. 


Paired Differences p-value 

Mean Std. 95% Confidence 

Deviatio Interval of the 
n Difference 
Lower | Uppe 
r 
Rater | prlHolistic 
1 Pair | Sentence Structure 
3 and Context - - 

-. 5467 -.537 .002 
pol Holistic eG, a a 1292 wo 
Sentence Structure 
and Context 

Rater | pr2Holistic 
2 Sentence Structure 
and Context — 7 
Pai -.4 5632 -.61 .001 
a po2Holistic a a pee .1897 io 
Sentence Structure 
and Context 


Table 5: Paired Test for Variable 3, Both Raters 


Results for variable six that assesses essay form correctness seem to be of a better value 
compared to variable three. The p-value of the paired test denotes a significant statistical 
difference between the results of the pre- and posttests for both raters, p= .000 and .000, 
which again rejects the null hypothesis. Unlike variable three, the effect size d= 0.94 for the 
first rater, and 1.08 for the second. The d-value cited suggests that the intervention seems to 
have a medium effect size when it comes to improving essay structure. Taken together, the 


effect size of the two variables suggests that experimental group seem to have improved 
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slightly in terms of the overall grammatical accuracy of their written production, and 


moderately in terms of their control over essay structure. 


Paired Differences p-value 
Mean Std. 95% Confidence 
Deviatio Interval of the 
n Difference 
Lower | Uppe 
r 
Pair prlEssay Structure 
6 - polEssay -.6667 8023 -.9662 000 
3671 
Structure 
Pair pr2Essay Structure 
6 — po2Essay -.7333 8277 1.0424 | 4243 OOO 
Structure 


Table 6: Paired Test for Variable 6, both Raters 


4.2.3 Results for research question three 


Research question three can be answered by analyzing variables four and five that indicate 
the use of a major academic writing characteristic namely, hedging, and the salience of 
scholarly academic tone described as the formal, impersonal, objective voice that is 
characteristic of academic writing. Scholarly academic voice avoids the use of personal or 
emotional language, over-sweeping adjectives, and colloquialism. As illustrated in tables 7 
and 8, the results of the paired test for the two variables by the two raters show a p-value of 
001 denoting significant difference between the results of the pre- and posttests, yet at a 
small effect size, where d= .6 and .65 for variable five, and .76 and .49 for variable six. These 
results indicate that the intervention may have led to a slight improvement in the scholarly 


voice of the experimental group. 
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Paired Differences p-value 
Mean Std. 95% Confidence 
Deviatio Interval of the 
n Difference 
Lower | Uppe 
r 
Pair prlUse of 
4 Cautious 
Language - - 
-.4667 .6814 -.7211 001 
polUse of Re oe 2122 ve 
Cautious 
Language 
Pair pri Scholarly and 
5 Academic Tone - 7 
-. .7761 -.8231 .001 
polScholarly and 8 : oee 2435 Of 
Academic Tone 
Table 7: Paired Test for Academic Tone-related Variables Rater | 
Paired Differences p-value 
Mean Std. 95% Confidence 
Deviatio Interval of the 
n Difference 
Lower | Uppe 
r 
Pair pr2Use of 
4 Cautious 
Language — - 
ae .7761 -.8231 .001 
po2Use of ae e eae 2435 ue 
Cautious 
Language 
Pair pr2Scholarly and 
5 Academic Tone — 7 
-. £7022 -.5622 02 
po2Scholarly and aul ? e .0378 U- 
Academic Tone 


Table 8: Paired Test for Academic Tone-related Variables Rater 2 


46 


4.3 Discussion 


The results of the experimental group can be categorized into three main areas, lexical, 
grammatical, and academic/scholarly. In the discussion section, each set of results shall be 
interpreted in the light of the findings of previous studies and the results of the control group 
with the purpose of explaining the meaning of the obtained numerical observations in the 
context of computer assisted language learning and academic writing pedagogy domains. The 
influence of the research setting will be referenced as a major determinant of the results 
obtained in terms of the prevailing L2 pedagogical environment, experimental group’s 


engagement, and theoretical and methodological maturity of the implemented intervention. 


4.3.1 Lexical effect of the intervention 


The results of the experimental group showed high efficacy in the recollection and 
employment of target academic lexical items d= 1.6. This level of lexical engagement was 
supported, albeit at a medium effect size d= 1.14, by the ability to use target academic 
vocabulary and formulaic language in correct collocation and colligation patterns. This 
observation denotes that learners seem to have managed to acquire lexical knowledge that is 
not strictly word-based, and lends empirical evidence to the assumption that the treatment 
group learned academic lexical sequences that are rich in academic patterns and collocations, 


yet rather rudimentarily. 


It is important to note here that the empirical research on the efficacy of CALL in vocabulary 
instruction is predominantly computer-focused. That is to say, incorporating CALL and 
MALL is hardly attempted in the available literature. Therefore, comparing the efficacy of 
the present study to that of previous research would entail synthesizing results from two 
separate lines of research. With that in mind, the results of the two variables being discussed 


seem to correspond to and compliment empirical findings in the available literature. To start 
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with, the d-value of the two variables being studied is quite comparable to the effect sizes 
attained in the literature of CALL-based vocabulary instruction. In his meta-analysis of 
sixteen studies, Chiu (2013) cites a large effect size d= 1.57 for CALL-based vocabulary 
instructions that are one month or less in duration, a large effect size d= 1.03 for university 
and secondary education participants compared to participants in elementary stages, and a 
large effect size d= 1.11 for not including games in the intervention. The current study was 
four weeks in duration; it targeted university students, and did not involve educational games. 
Accordingly, the reached effect size of the current treatment seems to be consistent with the 


overall efficacy of the sixteen studies of the meta-analysis despite the different contexts. 


It is distinctly important to note here that the seemingly insuperable language difficulties 
holistically manifested by Saudi university students cited in Alfadda (2012) and Alrabai 
(2016), and represented by the treatment group seem to have been partly resolved by using 
MALL represented by using mobile text messages to complement CALL instructions. 
Consistent with assertions made by Li et al. (2017), including a MALL activity in the 
vocabulary intervention seems to have resulted in decreasing the cognitive load of the newly 
introduced lexical items, both individual words and formulaic chunks, and created what 
might be termed a “cognitive continuum” with multiple input channels that allowed frequent 
exposure to target lexical items. That continuum may have accelerated the internalization of 
target lexical items, hence their strong presence in the posttest results. The results of the 
intervention’s lexically related variables; therefore, are comparable to the results of other 
empirical research in the available literature. The positive students’ attitude towards using 
mobile applications for texting as a medium of vocabulary instruction cited in Lu (2008), 
Cavus and Ibrahim (2009), and Gurock (2016) seems to have constituted a pedagogical value 


that positively influenced the acquisition of academic lexical items. 
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On a similar note, the meaningful effect size rendered by the present study resonates with the 
quantitative results of Sung et al. (2015) meta-analysis that measured the overall 
effectiveness of MALL through scrutinizing the results of 43 journal articles and 2 doctoral 
dissertations. Indeed, the largest proportion of studies included in the meta-analysis sampled 
elementary school students, yet a substantial 26.7% of subjects were college students, the 
most frequent treatment duration was 1-6 months, most studies 73.3% used handheld devices, 
and the most studied skill was vocabulary. The above-mentioned criteria imply that the 
present study has been conducted under conditions well within the meta-analysis’s scope of 
variables, accordingly, the conclusions arrived at by the present study can be compared to the 
results of the meta-analysis. The findings of the meta-analysis suggest a moderate effect size 
for MALL; that is 70.7% or learners who used mobile devices performed significantly better 
than those who did not use them. A result that may clarify the high lexical effect size of the 


present intervention. 


Comparing the mean scores for lexical development of both experimental and control groups’ 
performance reveals that given the exact learning circumstances apart from the intervention 
procedures, significant differences can be seen between the control and treatment groups. 
Despite the fact that the mean scores of the treatment group for the two lexical variables were 
significantly less than the control group’s mean scores for the same variables in the pretest, 
the mean scores for the treatment group were markedly superior to the scores of the control 


group in the posttest, suggesting significant gains for the intervention. 
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Variable 1 Target V Count 


Variable 2 Target V Usage 


Mean Pretest 


Mean Posttest 


Mean Pretest 


Mean Posttest 


Rater 1 


0.25 


0.75 


Rater 2 


0.25 


0.75 


Table 9: Mean Scores for Control Group’s Vocabulary-related Variables out of 3 Points per 


Variable 


Variable 1 Target V Count Variable 2 Target V Usage 
Mean Pretest Mean Posttest Mean Pretest Mean Posttest 
Rater | 0.13 1.36 0.23 1.5 
Rater 2 0.13 1.36 0.23 15 


Table 10: Mean Scores for Treatment Group’s Vocabulary-related Variables out of 3 Points 


per Variable 


4.3.2 Grammatical effect of the intervention 


Despite the fact that English grammar was not explicitly taught in the present intervention, 
research assumptions were formulated to anticipate progress on the grammatical-structural 
level based on the density and variety of target lexical items. There was no sufficient 
empirical evidence in available literature that supports acquiring grammatical skills using 
CALL, albeit Nutta (1998), however, a modern language acquisition theory, namely usage- 


based learning (Beckner et al. 2009) stipulates that lexicon and grammar are highly 


50 


intertwined rather than separate, hence the assumption that incidental grammatical 


development might occur. 


The results pertinent to grammatical development seem to echo the cited pedagogical 
challenges Saudi second language learners are facing. The existing educational limitations do 
not seem to impede the transition from L2 basic interpersonal communicative skills to L2 
cognitive academic language proficiency, but rather the progress from L1 basic interpersonal 
communicative skills to L2 basic interpersonal communicative skills, including academic 
language competence. The problem of using LI as a medium of instruction as cited in 
Alarabi (2016), for instance, minimizes exposure to L2 lexico-grammatical components, and 
undermines the associated cognitive processes, and consequently impairs the competence 
required to engage L2 at an appropriate level of grammatical accuracy. The effect of this 
widespread practice among other administrative and cultural factors (Alarabi 2016) seems to 


be observable in the results of the present study. 


In order to measure grammatical and structural gains of the intervention, grades of variables 
three and six were analyzed. It can be observed that there seems to be a significant statistical 
difference between the pretest and posttest results for variable three p= .001 and .002 for 
raters one and two, and for variable six p= .000 and .000 for raters one and two. However, the 
actual effect size seems to range between small and medium compared to L2 research 
benchmarks. To elaborate on this point, the paired raters’ scores illustrated in Table 11 
indicate similar d-values for variable three, namely Holistic Sentence Structure, and 


dissimilar effect sizes for variable six; Essay Structure. 
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d-value Holistic d-value Essay 
Sentence Structure 


Structure 
Rater 1 0.59 0.94 
Rater 2 0.77 1 


Table 11: Effect Size for Grammar-related Variables per Rater 


A high level of linguistic deficiency was anticipated for all participants in the review of 
literature, which is why target lexical items and their grammatical environments were 
presented in the form of contextualized essays rather than individual word lists. Effort was 
exerted during the intervention to emphasize the importance of collocation patterns, and the 
inseparability of form and meaning. The exercises and text messages were also carefully 
designed to present and assess a comprehensible input that samples a moderate level of 
academic production. Although the paired test results seem to show significant difference 
between the grades of the pre- and posttests of the treatment group, the actual effect size that 
occurred is small for sentence structure and small/medium for essay structure. The researcher 
believes that the immediate reason for this limited effect size is students’ linguistic 
incompetence manifested in the pretest essays. The pretest results for both control and 
treatment groups indicated a limited and vastly grammatically flawed written production. The 
word count of the pretest essays could be as less as ten words in some cases, and they entirely 
lacked structure and form, i.e. no introduction, thesis statement, topic sentences, supporting 
details, or conclusions. Some pretest essays comprised only simple mind maps and diagrams 
of fragmented ideas with little or no substantial content. Therefore, there was indeed a visible 
difference in the form and structure of essays in the posttest since the treatment group were 
incidentally introduced to the different parts of the formal essay, and a brief explanation was 


provided when necessary. 
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Comparing the performance of the pre-and posttest results of the treatment and control 
groups (Tables 12 and 13) reflects better gains for the treatment group for the grammar- 
related variables. The probable reason for such linguistic disparity, though marginal in both 
cases, is twofold. The first, academic writing per se, as an independent writing register with 
conventionalized lexico-grammatical features is out of Saudi university-level education’s 
focus, which suggests lack of teaching approaches and curricula that introduce general 
academic phraseology. The second, there is the learners’ disinterest in; hence demotivation to 


interact with traditional methods of teaching, which do not involve the use of technology. 


Variable 3 Holistic Sentence Variable 6 Essay Structure 


Structure 


Mean Pretest Mean Posttest Mean Pretest Mean Posttest 
Rater 1 0.1 0.4 0.9 1.6 
Rater 2 0.6 0.4 0.8 1.5 


Table 12: Mean Scores for Treatment Group’s Grammar-related Variables out of 3 Points per 


Variable 


Variable 3 Holistic Sentence 


Structure 


Variable 6 Essay Structure 


Mean Pretest Mean Posttest Mean Pretest Mean Posttest 
Rater 1 0.0 0.0 0.5 0.5 
Rater 2 0.0 0.0 0.5 0.5 


Table 13: Mean Scores for Control Group’s Grammar-related Variables out of 3 Points per 


Variable 
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4.3.3 Scholarly voice effect of the intervention 


To discuss the intervention’s results for academic and scholarly voice, variables four and 
five, namely The Use of Cautious Language, and Scholarly and Academic Voice were 
employed. The use of cautious language or hedging means employing lexical devices such as 
perhaps, may, seems to etc., to limit the force of claims made by academic writers, and it is a 
dominant characteristic of academic language (Chen & Baker 2010, Hinkel 2005). Scholarly 
voice was defined for the two raters of the present study as the formal, impersonal, objective 
voice that is characteristic of academic writing. It avoids the use of personal or emotional 
language, Over-sweeping adjectives, and colloquialism. Accordingly, the two raters had a 
clear understanding of what to evaluate in the pre- and posttests. The paired test refers to 
statistically significant difference in the performance of the experimental group where p= 
.OO1 for both raters for hedging and .001 and .026 for scholarly voice. This relative statistical 
significance can only be adequately interpreted in the light of the generated effect size. The 


d-value for the two variable (Table 14) indicate small effect size for the treatment group. 


d-value Use of d-value Use of 

Cautious Language Scholarly Voice 
Rater 1 0.6 0.7 
Rater 2 0.6 0.4 


Table 14: Effect Size for Academic Voice-related Variables per Rater 


Once again, very scarce empirical research in available literature, outside data-driven 
learning, that examines the use of academic formulaic sequences in the context of CALL and 
MALL to introduce a change in EFL students’ academic repertoire. The present study’s 
results; however, are consistent with the work of Gurocak (2016) who employed Coxhead’s 


(2000) academic word list and text messaging as the means of delivery in a short vocabulary 
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intervention that showed statistically significant gains for the experimental group. Results are 
also consistent with Li, Cummins and Deng (2017) who used frequent structured exposure 
via text messaging to highly salient vocabulary items from Cobb’s VocabularyProfilers 
academic wordlist and concluded that text messaging is “a malleable and gratifying means of 
vocabulary instructions “(P. 13). It could be noted here that the present study explores what 
may be considered a new territory for CALL where basic academic features and phraseology 
are introduced to EFL students. Its effect size, while small, may represent a significant 
addition to the available body of research available on the topic. Examining the mean scores 
of the control and treatment groups for academic voice-related variables may shed more light 


on this assertion. 


Variable 4 Use of Cautious Variable 5 Scholarly and 
Language Academic Voice 
Mean Pretest Mean Posttest Mean Pretest Mean Posttest 
Rater 1 0.86 1.33 0.56 1.1 
Rater 2 0.86 1.4 0.53 0.83 


Table 15: Mean Scores for Treatment Group’s Academic Voice-related Variables out of 3 


Points per Variable 
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Variable 4 Use of Cautious Variable 5 Scholarly and 
Language Academic Voice 
Mean Pretest Mean Posttest Mean Pretest Mean Posttest 
Rater | 0.5 0.5 0.25 0.75 
Rater 2 0.5 0.5 0.25 0.5 


Table 16: Mean Scores for Control Group’s Academic Voice-related Variables out of 3 


Points per Variable 


Examining tables 15 and 16 reveals that the control group did not register significant gains in 
the two variables under scrutiny compared to the treatment group. These results may imply 
that while it might be known to language instructors who teach using non-computer-assisted 
methods, register-specific phraseology of academic writing is nowhere to be found in Saudi 
EFL curricula. It can also imply that not using technology in EFL instructions for the control 


group of the present study foreshadowed their underperformance. 


4.4 Qualitative Findings and Final Comments 


Consistent with cognitive psychology research, the results of the present study suggests that 
quick and ubiquitous complimentary activities such as text messaging in the case of this 
intervention that prompt the explicit use of vocabulary can increase the efficacy of 
vocabulary instructions introduced by CALL, as they allow several opportunities for 
vocabulary recollection. The large effect size of all intervention’s variables together d= 1.37 
for both raters suggests that the combined pedagogical forces of computer and mobile 
assisted language learning may enable a more effective retention of academic vocabulary and 
phraseology. It was also found that the intervention addressed one of the major problems 


faced by Saudi EFL students, namely linguistic disinterest, since it stimulated students’ 
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intrinsic motivation, and allowed them to experience academic language instructions, perhaps 


for the first time, using a cognitively stimulating technology-based pedagogical model. 


Also consistent with modern pedagogical research, selecting academic vocabulary 
from an academic word list was positively viewed by participants in the treatment group, did 
prove to adequately prioritize vocabulary choice in a manner that allows instructions to be as 


cognitively beneficial as possible. 
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Chapter five: Conclusion 


5.1 Summary of the Research 


This study aimed to investigate the pedagogical value of incorporating computer and mobile 
assisted language learning to teach basic academic writing skills to pre-intermediate 
university students in the challenging English as a foreign language environment of Saudi 
Arabia. To that end, an intensive four-week intervention was designed observing current 
CALL and MALL research trends where basic academic language vocabulary, phraseology, 
and lexico-grammar were introduced using digital flash cards afforded by Quizlet platform. 
The intervention’s target lexical items were afforded by Oxford Phrasal Academic Lexicon 
(OPAL), a cognitive wordlist that has been recently compiled where highly salient core 
academic vocabulary that are not domain specific were grouped in sub-lists based on their 
frequency. Four very frequent, hence relatively easy sub-lists of ten words each were selected 
to be taught in order to match the linguistic level of the experimental cohort. Moreover, short 
lexical chunks that are characteristic of academic language afforded also by OPAL were 
selected to be taught in the intervention along with one major conventionalized feature of 
academic language, namely hedging, i.e. the use of cautious language. Four essays were then 
tailored to contextualize all target lexical items so that every sub-list of ten words and a 
number of academic formulaic items were contextualized per essay to facilitate their 


presentation to learners. 


Following to selecting and contextualizing target language, necessary formal permissions to 
conduct the experiment and collect data in a Saudi university commenced. Once approved, 
thirty participants for the experimental group and four for the control group were 
heterogeneously selected to represent a specimen of Saudi pre-intermediate English language 


learners. Data were collected using pre-/post-test format; accordingly, the two groups were 
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instructed to write a short essay of 250-300 words then experimental group started the four- 
week CALL-MALL intervention, while the control group started the university’s regular 
four-week intensive preparatory course. A grading rubric was designed and calibrated for 
measures of reliability and validity to assess six particular variables that were intended to 


confirm of negate the following research questions: 


1. Can Computer Assisted Language Learning (CALL) improve pre-intermediate Saudi 
university students’ academic vocabulary in terms of quantity and quality? 

2. Can CALL improve the grammatical and/or structural features of pre-intermediate 
Saudi university students’ academic writing? 

3. Can CALL improve the overall academic voice in the written production of pre- 


intermediate Saudi university students? 


Accordingly, it was hypothesized that: 


1. Using digital flash cards afforded by the online platform Quizlet, and the academic 
vocabulary and phraseology provided by the web-based Oxford Phrasal Academic 
Lexicon (OPAL) enhanced experimental group’s academic vocabulary, and afforded 
more reliable vocabulary internalization and operationalization measurable by 
analyzing the results of the posttest. 

2. The combined pedagogical forces of computer and mobile assisted language learning 
enabled a more effective retention of academic vocabulary and phraseology. 

3. The academic voice installed in the lexico-grammar of academic language was 


consequently enhanced through the use of impersonal and cautious language. 


Post-test results were collected by means of short essay writing in a topic that is similar but 
not identical to that of the pre-test to control for task complexity and language variations. 


After collecting data, two experienced raters were assigned by the Saudi university to grade 


59 


the results of the pre- and post-test results. Intra- and inter-rater reliability assessments were 
conducted to ensure the validity of the grading process, and then clear detailed grading 
instructions were provided to both raters in order to guarantee the highest level of consistency 
and to ensure that raters are not distracted by any writing features that are not measurable by 
the rubric. The results of the two raters’ grading were then coded and analyzed using IBM 
SPSS statistics version 24 for windows, then the intervention’s effect size was calculated 


separately using David Wilson’s calculator. 


Results of the ¢-test for each pair of the six variables represented an answer to one of the three 
research questions. For vocabulary related variables, a statistically significant difference 
between the pre- and post-test results was observed with a large effect size (d= 1.6). As for 
grammar and structure-related variables, and academic tone variables, and while statistically 
significant differences between the results of pre- and post-tests were observed, small to 
medium effect size (d= .57) and (.76) respectively. The research findings suggest that the 
proposed CALL-MALL intervention seem to have affected a large difference in terms of the 
use of academic vocabulary, and smaller gains in terms of grammatical and the overall 
academic voice of the experimental group. When comparing the results of the experimental 
and control groups; however, the intervention shows to have caused higher gains compared to 


the traditional textbook method. 


5.2 Pedagogical implications 


It seems that based on the findings of the present study, the combined pedagogical forces of 
CALL and MALL have the potential to overcome the characteristic linguistic incompetence 
of Saudi university students. Accordingly, the present study may have the following 


academic L2 teaching implications. 
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Firstly, pedagogical academic word lists can be a valuable asset to prioritize vocabulary 
teaching in the Saudi context. There could have been skepticism about the value of 
pedagogical word lists in an L1-dominant L2 classrooms, but when allowed the chance, 
learners tend to take interest in, and capitalize on the use of contextualized words from an 
academic vocabulary list as a way to understand academic language conventionalized 


patterns without interference from teacher intuition. 


Secondly, designing an English for Academic Purposes (EAP) course that presents a 
balanced technological, pedagogical, and content knowledge may integrate CALL in EFL 
settings in a way where the use of computers complements rather than supplement language 
instructions so that the learning experience is enhanced with the resources and opportunities 


made available by computers and would not be available otherwise. 


Finally, MALL can prove to be a valuable supplement to a CALL-based EAP course, 
especially if used for texting target lexical items at regular intervals. While vocabulary 
teaching mobile applications are available for public use, texting seem to have a significant 
effect in the Saudi context, especially because the use of smart mobile phones is part of the 


Saudi culture. 


5.3 Research Limitations 


While the study showed valuable gains for CALL, a number of design concerns should be 
taken in consideration while interpreting its results. Because of cultural constraints, and to 
abide by ethical guidelines of research, only thirty participants sat for the pre-, post-tests, and 
attended the four intervention sessions. This number of participants, while produced 
statistically significant results may not allow much generalization latitude. On a similar note, 
the number of the control group was very small and may not adequately reflect a realistic 


statistical value for the scores of its participants. 
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On a different note, the number of lexical items selected was limited to forty vocabulary 
items and less than ten formulaic language items. While this small number of lexical items 
was dictated by the linguistic level of participants, the duration of the intervention could have 
allowed more target vocabulary, which could have increased the pedagogical value of the 
intervention. These limitations, while true, should not obscure the significant effect size 
produced by the study, and should allow the findings of the present research to be viewed 


through a fairly positive lens. 


Along similar lines, the treatment group comprised pre-intermediate university students. 
While the intervention yielded a generally acceptable effect size, participants with higher 
level of English language could have produced higher effect size. Explaining the notion of 
academic vocabulary to struggling L2 students constituted a large cognitive load that required 
long processing time on their part, which led to small gains on the academic voice variables, 
as participants did not succeed in recognizing and appropriately using the criterial features 
and conventions of academic prose. This is because they did not have the lexical or 
grammatical control required for academic writing. Therefore, raters seem to have perceived 


participants’ written production as unstructured and/or overly personal. 


Similarly, the low L2 proficiency level of participants limited not only the number of taught 
lexical items, but also the range of taught academic writing skills. For instance, it was not 
possible to introduce important academic writing skills such as paraphrasing because the 
lexico-grammatical repertoire of participants did not allow such sophisticated level of 
instruction. The vocabulary teaching; therefore, sounded quite mechanical and was 
maintained at a rather rudimentary level, where further development of participants’ lexical 


fluency was not possible. 
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Finally, in terms of design, on the one hand, the pedagogical value of CALL and MALL in 
teaching academic vocabulary was not put to test in a longitudinal format. That is, there was 
no delayed post-test to measure participants’ retention learning of taught lexical items and 
academic features beyond the initial post-test. Hence, there is no evidence that the 
intervention employed in this study could prove effective on a longer scale. On the other 
hand, relying solely on one web-based application during in-class vocabulary instruction, 
namely, Quizlet created a rather redundant and sometimes monotonous atmosphere, and to 
that end, a variation in the used CALL applications could have created a more engaging 


vocabulary-learning environment. 


Future research directions suggested by the present study point towards more synthesis of 
computer and mobile assisted language learning tools and methods in Saudi higher education 
contexts. Future research endeavours could focus studying the possibility of using CALL- 
based methods to teach academic grammatical features without which Saudi university 


students may be unable to develop a full range of academic competence. 
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* Human data may be defined as non-numerical information (e.g. opinions, preferences, responses, ideas) collected from human 
beings through structured or unstructured methods (e.g. interviews, surveys, questionnaires or focus groups) 


**Adult at risk is defined by the Department of Health as 'a person aged 18 years or over who is or may be in need of 
community care services by reason of mental or other disability, age or illness; and who is or may be unable to take care of him 
or herself, or unable to protect him or herself against significant harm or exploitation’, Department of Health paper, 'No Secrets’ 


https://www.gov.uk/government/publications/safeguarding-policy-protecting-vulnerable-adults/sd8-opgs- 
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3. Project Details 


Project Title: Investigating the Efficacy of Flexible Language Acquisition Project (FLAX) in 
Developing Academic Writing: The Case of Saudi University Students 


Project Start date: DD/MM/YYYY Expected End Date DD/MM/YYYY 


Project Outline: 


Give a brief overview - Recommended: list bullet points. For examples see sample application form 


Subject area and background of the study: 


> 


> 


VV V 


The study intends to explore the difficulties in academic writing in both individual words and 


among university Saudi students 


As an English language teacher in Saudi university, I noticed that it is a common difficulty for 
Saudi students to write academic English, so it is very important for me to investigate what the 
difficulties are in order to help them to learn better. Also, this study will help me finish my MA 
dissertation. 


The research will be conducted in Kingdom of Saudi Arabia. 
This research is not being funded. 
This study will consist of the whole body of MA study. 


Purpose, aims and objectives of the research: 


> 


VV V 


The purpose of this study is to investigate the extent to which Computer-assisted Language 


Learning (CALL) can develop the lexico-grammatical competence of university-level Saudi 
students. The research intends to answer the two following research questions: 


o How can corpus-based instructions, such as an application of Computer-assisted Language 


Learning (CALL) helps Saudi students’ improve academic writing? 
o Is using a corpus-informed web interface that is supported by mobile phone applications 


capable of developing the lexical richness of Saudi English as a foreign language (EFL) 


university students’ academic writing? 
The participants will be recruited from Saudi university students who study English in English 
language department with age range from 18-30, on both undergraduate and postgraduate levels. 
This study will take place in a university in Saudi Arabia. 
The data collected will be secured in password-protected file. 
The study aims to answer the research questions. 
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Brief description of research procedures (methods, tests etc): 
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Section D: Ethical Issues Checklist 


This section MUST be completed by the applicant if they marked YES to one or more questions in section B 
Once you have completed this section please complete the rest of this form. 


For sample forms, document templates and additional information relating to any of the supporting documents listed 
below please refer to faculty templates and guidance located at the ADH Ethics website 


4.1 Principal data collection methods SUPPORTING 
DOCUMENT REQUIRED 


Please tick V as applicable for all of the relevant 


methods and refer to notes highlighted in the right If you mark YES to any of the 


data collection methods within 

question 4.1 please provide 
Questionnaires copies of the Participant 
Audio/video recordings Information Sheet and 
Participant Consent Form 
: (using the templates 
CSE TENIENS available on the ADH Ethics 
Focus groups/workshops website) for assessment and 
Documents/archives submit with your application. 
Please note that 
questionnaires and question 
lists are not required for 
FREC/DRESC assessment but 
must be referred to their Tutor 


Interviews 


Online surveys 


Other (please § Comparative test : essay writing 
briefly specify) 


4.2 Anonymisation of data 


Please tick V as applicable for the following question 


Will you be anonymising the data collected from participants? 


If you mark YES please All participants will be unidentified to ensure confidentiality and securing participants 
briefly state how and why personal information, they will be identified as participant A, B, C. 

you will be anonymising that 

data. For examples see 


sample application form 


4.3 Working with children/ adults at risk 


Will you be observing, interacting or otherwise with the following participants? SUPPORTING 


Please tick VY as applicable all of the following questions YES NO DOCUMENT 
and refer to notes highlighted to the right REQUIRED 


If you mark YES within 
question 4.3 please provide 


Children (under the age of 18) v 
v 
7, copies of the copies of the 
v 


Adults without capacity to consent 


Those with learning disabilities 
Adults at risk 


Parent/Guardian Consent 
Form (using the 
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Other 
(please 
briefly 
specify) 


4.4 Injury risk declaration 


Please tick V as applicable for the 
following question and refer to notes 
highlighted to the right 


Will you be researching into activities that 
may involve a risk of personal injury to the 
participants? 


4.5 Human behaviour declaration 


templates available on 
ADH Ethics website) for 
assessment and submit 
with your application 


SUPPORTING DOCUMENT REQUIRED 


If you mark YES to question 4.4 please provide a 
DMU Risk Assessment Form and submit with 
your application - Please liaise with your tutor and 
consult with your Faculty Health and Safety 
Officer for this form 


Please tick V as applicable for the following question 


Will your research be supporting innovation that might impact on human behaviour? 


e.g. Behavioural Studies and/or Activities which may pose a physical risk to other people not 


If YES please briefly 
specify. For examples 
see 


sample application form 


4.6 Environmental risk declaration 


Please tick V as applicable for the following 
question and refer to notes highlighted to the right 


SUPPORTING DOCUMENT 
REQUIRED 


Will your research involve any activities which may pose 
a risk to the environment using a potentially hazardous 
substance? 


If you mark YES to question 4.6 
please provide a DMU COSHH 
Form and submit with your 


If YES please briefly 
specify. For 
examples see 


sample application 
form 


4.7 Sensitive Research declaration 


application - Please liaise with your 
tutor and consult with your Faculty 
Health and Safety Officer for this 
form 


Please tick V as applicable for the following question 
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Will you be researching topics that are concerned with the following ‘sensitive research’ areas: 
illegal activities, including the collection of source data, e.g. access to web sites normally 
prohibited on university servers, or extremism and radicalisation, pornography etc.? 

If YES please 

briefly specify. 


For examples see 
sample 


application form 


Section E: How Ethical Issues Will Be Addressed 


This section MUST be completed by the applicant if they marked YES to one or more questions in section B 
Once you have completed this section please complete the rest of this form. 


IMPORTANT NOTICE: Please refer to the ADH Sample Application form for possible examples available on the 
ADH Ethics website 


5. Ethical Issues — Please state briefly all ethical issues identified, how they will be addressed and describe the 
methods that you will use 


(Recommended: list bullet points) 


e This study will involve human-being who will be informed with full details of the study and what | am going to do with 
the collected data, to address ethical issues. 

e I will provide the participant/organisations with full details of the study in non-specialist language, using the faculty 
approved Information Sheet (this will allow participant to make an informed decision whether to take part or not) and I 
confirm in writing if the participant: is willing to take part; agrees to their data to be used; acknowledges that they have 
read and understood the proposed study by completing a consent form. 

The participants will be informed that participation is voluntary and granted the right and freedom to withdraw from the 
study. 
The research will take place in a university in Saudi Arabia. 


Any information will be collecting in class will be kept confidential. 


I will minimise personal risks to themselves, No information will be given in dissertation that can be linked back to 
participants. 


Translating the information into Arabic, when the participants do not understand. 


Collected data in the form of expository essays will be stored in a password protected computer in PDF file. 
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Section F: Ethical References and Additional Factors 


This section MUST be completed by the applicant if they marked YES to one or more questions in section B 


Once you have completed this section please complete the rest of this form. 


6. To which ethical codes of conduct have you referred? COMPULSORY 


This section MUST be 


Please tick V as applicable all of the following questions completed as a 
COMPULSORY 


a) | confirm that all information collected will be processed by REQUIREMENT 
use in accordance GDPR 2018 


By government law all data 


b) | confirm that | will follow DMU’s ethical codes of conduct must be handled in 
for Good Research Practice accordance with GDPR and 
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c) Are there any other ethical codes not listed above that you v all researchers must 
will be referring to? comply to DMU’s 
guidelines for Good 


If YES please specify and include any links below: Rascarch Practice 


Web address: 


7. Additional factors 


Please tick V as applicable for the following question 


Are there other additional factors that could/will give rise to ethical concerns e.g. communication 
difficulties? 


If YES please specify. e Because participants are not native English speakers, consent forms will be 
For examples see bilingual; Arabic and English, in order to allow informed decisions on the 
sample application part of the selection cohort weather or not to participate in the study. Both 
foun Arabic and English texts will have the exact same meaning. 


Section G: Questions for Ethical Approval of Sensitive Research 


This section MUST be completed by the applicant if they marked YES to one or more questions in section B 
Once you have completed this section please complete the rest of this form. Further guidance can be located 
at: http://www.dmu.ac.uk/research/ethics-and-governance/sensitive-research.aspx 


ALL RESEARCH WHICH ANSWERS YES TO ONE OR MORE QUESTIONS IN SECTION G IS CLASSED AS 
HIGH RISK AND MUST BE REFERRED TO ADHethics@dmu.ac.uk FOR CONSIDERATION BY THE FACULTY 
HEAD OF ETHICS ONCE FULLY COMPLETED AND AUTHORISED BY THEIR TUTOR 


8. Sensitive Research 


Please tick V as applicable all of the following questions YES NO 
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1. Does the intended research include research into illegal activities? (This may include, v 
but is not limited to, research into hate crime, theft, fraud, or harmful and illegal cultural 
practices, etc.). Please note: the university does not permit any crime to be 
committed for research purposes, such as accessing images of paedophilia or 
child abuse, unless special permission has been granted by the Home Office. 


2. Does the research involve deception? (if yes, please give brief details as to why/how in v 
the space below) 


3. Will the research require the use of sites usually prohibited on university computers v 
(e.g. pornography or the sites of these prohibited organisations)? 
4. Has the research been commissioned under a commercial contract requiring secure v 


storage for related materials? 
5. Does the intended research fit into any of the following categories? If so, please indicate 
which in the points below: 
a) Terrorism, extremism, terrorist or extremist organisations or groups, extremist 
ideologies, radicalisation, de-radicalisation 


b) Commissioned by the military or GCHQ: v 
c) Commissioned under an EU/US security call or similar: v 
d) Involve the acquisition of security clearances (including the Official Secrets Act): v 


Please tick VY as applicable the following question and refer to the note highlighted 
below 

Will your research involve visits to websites that might be associated with radicalisation or 
terrorist/extremist organisations or groups? 


YES NO 


If you answer YES to question 9 you are advised that such sites may be subject to surveillance by the police and 
accessing those sites might lead to police enquiries. It is strongly recommended that you use your university 
network address, once you have received ethical approval, which will ensure these activities are flagged as a 
legitimate part of your research. Whilst acquiring ethical approval for this project and adhering to University 
guidance on accessing websites and storing related materials securely will allow the University to verify the 
legitimacy of you accessing these websites, it cannot guarantee legal protection. 


If you answered YES please acknowledge that you understand this risk by marking | Agree 
the ‘I Agree’ box with a tick (v) 


10. Storage and Transmission of Research Materials 

The secure storage of data and research material is strongly recommended to all who answered YES to section 8 
question 5 (although all researchers may make use of the Information Media Technology Services (ITMS) 
provisions detailed in this form). Please note that anyone storing participants’ personal data is subject to separate 
legislation and requirements. Details are outlined here, and in the university's Research Records Retention Policy. 


Please tick VY as applicable for the following questions and refer to notes highlighted 
below 

Does your research involve the downloading and storage on a computer of any materials v 
relating to extremism or radicalisation (for example, records, statements or other documents)? 
If you answered YES to question 10, you should request a secure file share from ITMS to be created for your 


YES NO 
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project, with access restricted to you, or if absolutely necessary, any internal co-investigator(s). The research 
materials should not be kept on a personal computer, and all online research in this area should be done on 
university servers. Physical data should be scanned and uploaded to the password-protected server; where this is 
not possible, it should be kept in a locked filing cabinet or similar on university premises. 


You will need to agree to store all materials relevant to question 9 and question 11, as well as any other materials 
related to your research project in accordance with this advice in order to gain ethical approval. 


If you answered YES please confirm you will store a// research documents in 
accordance with this advice by marking the ‘I Agree’ box with a tick (v) 


11. Storage and Transmission of Research Materials 


Please tick VY as applicable for the following question and refer to the note highlighted 


below NO 


Might your research involve the electronic transmission of such materials to project Co- v 
Investigators? 


NOTE: The Terrorism Act (2006) and the Counter-Terrorism and Security Act (2015) outlaw the dissemination 
of terrorist publications if the individual concerned has the intention to encourage or induce others. Publications 
disseminated for the purposes of an approved and clearly defined research project should not amount to an 
offence, because the requisite intention is unlikely to be present. However, you are advised to exercise caution 
and avoid dissemination of raw research materials where possible. 


If you answered YES you will need to agree to only transmit these materials to Co- | Agree 
Investigators after they have been password-protected and that you will only use 

‘Zend’, which encrypts materials in transmission by marking the ‘I Agree’ box with a 

tick (Vv) 


DEFINITIONS 

Illegal activities incorporates any illegal activity; for example, trespassing, theft, or online piracy 

Hate Crimes are those committed against someone because of their disability, gender-identity, race, religion or belief, or sexual 
orientation. 

Harmful and illegal cultural practices: these include violence against women and girls, Female Genital Mutilation (FGM), 
forced marriage, child sexual exploitation and honour-based violence. 

Accessing prohibited websites: You will need to seek permission from ITMS; advice on how to gain permission is available 


from the ITMS helpdesk. 

Radicalisation refers to the process by which a person comes to support terrorism and forms of extremism leading to terrorism 
De-radicalisation usually refers to activity aimed at a person who supports terrorism and in some cases has engaged in terrorist 
related activity, which is intended to effect cognitive and/or behavioural change leading to a new outlook on terrorism and/or 
disengagement from it. 

Secure File Share: You will need to ask ITMS to create a Secure File Share for your project, with access restricted to yourself, 
or if absolutely necessary, any internal co-investigator(s). Advice is available from the IT MS helpdesk. 


Zend: advice on using Zend is available from the TMS helpdesk. 


Section H: Authorisation and Approval 


BEFORE COMPLETING THIS SECTION PLEASE REFER TO THE NOTE BELOW: 


Applications must undergo formal assessment BEFORE undertaking any data collection directly 


associated with the proposed research activity 
Applicants must submit their fully completed application and supporting documents from an official DMU e- 
mail account to their Tutor for checking, authorisation and risk assessment in the first instance 
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Please consider naming and numbering documents appropriately to assist reviewers 

Accepted file formats: word.doc / word.docx — only electronic forms must be completed/submitted 

Signing Section H (12 — 13), confirms that the applicant and Tutor has read, understood and will/assist the 
applicant to comply with the ethical procedures stated in this application 

If you are requested to resubmit please use the same completed form and clearly track/highlight all changes 


ADH Ethics Overview Guide, sample application forms and supporting document templates and submission dates 
can be located the ADH Ethics website 


12. Applicant — Checklist and Authorisation 
Please check your application and tick V as applicable to the following questions YES NO 


a) | declare that | have completed all fields, understand and will comply with the ethical v 
procedures declared in Section A — H of this application form 


b) | confirm that | have consulted with the Sample ADH Ethics Application when v 
completing this form - available on the ADH Ethics website 

c) | confirm that | have referred this ethics application to my Tutor v 
d) | confirm that | have read the Responsibilities of the Researcher guidelines and | will v 


comply with them 


e) | confirm that | have completed Section G Questions for Ethical Approval of v 
Sensitive Research (COMPULSORY) 


f) | declare that | have considered the ethical implications of the research proposed in this v 
application and understand that | must not undertake any research activity until this form is 

fully approved 

g) If the circumstances of my study changes | agree to re-apply for ethical approval before v 
commencing with my research 


h) | confirm that | have submitted for assessment the following required supporting documents (listed below) with 
this application form. Please refer to section D of this form to ensure the correct documents are submitted 


Participant Information Sheet v Consent Form v 
DMU Health and Safety Assessment Form DMU COSHH Risk Assessment 
Form 


Applicant Name: DD/MM/YYYY 


HAIFA ALRESHEEDI 09/05/2019 
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13. Tutor — Authorisation 


BEFORE COMPLETING THIS SECTION PLEASE REFER TO THE NOTE BELOW: 


Tutors MUST confirm that they fully authorise the content of the application and supporting documents by 
completing this section in full. Forms must be checked and risk assessed in the first instance by the Tutor 
By completing this section the tutor confirms that they have read, understood and will support the applicant 


to comply with the above ethical procedures stated in this application 


Accepted file formats: word.doc / word.docx — only electronic forms must be submitted 
Sample application forms are available to assist you with your assessment 


Please tick V as applicable 


a) Risk Assessment: | declare the research proposed in this application to be: 


If LOW RISK* 
please tick (v) 


If MEDIUM RISK** 
please tick (v) 


If HIGH RISK*** 
please tick (v) 


b) | confirm that | have read and agreed the content of this form and fully support the research 


proposed in this application 


c) | declare that the applicant named above has completed their application in collaboration with myself 
as the applicant’s tutor and will be working under my supervision 
d) | confirm that the applicant has completed all fields in Section A — H of this application form 


e) | confirm that | have assessed and agreed the content of all supporting documents required as part 
of this submission (as indicated in Section D of this form) 


Jie Liu 


Title/Role: 


Low Risk* 


Anonymous questionnaires 
market or opinion research, 
customer satisfaction surveys, 
previously collected 
anonymous data held by the 
university which cannot be 
traced back to the individuals 
who provided them, 


observations of performances/ 
behaviour, service level 
assessments. (Provided that 
these do not touch on 
sensitive topics). Low risk 
tutor authorised forms are 
granted full approval and 
must be submitted by the 


Supervisor 


Date authorised: 


DD/MM/YYYY 
10/05/2019 


Date re-authorised: 


Medium Risk**Face-to-face focus 
groups, workshops and interviews, 
studies where anonymity cannot be 
maintained, collection of and/or 
research using human tissue/fluids, 
studies which involve participants under 
18 or adults at risk, studies that induce 
anxiety, stress or other harmful 
psychological states 


Medium risk Tutor authorised forms 
MUST undergo assessment by the 
relevant committee. Applicants must 
not conduct research of this type 
without written approval from the 
relevant committee. Applications must 
be submitted by the tutor (copying in 
the applicant) to one of the following 
relevant e-mail addresses: School of 
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Department/Module: 


TEFL5006 


High Risk*** Research topics 
concerned with the following 
‘sensitive research’ areas: illegal 
activities, including the collection of 
source data, e.g. ethics statistics, or 
access to web sites normally 
prohibited on university servers, or 
extremism and radicalisation. Studies 
which will need to be reported to 
University Research Ethics 
Committee (UREC). High risk tutor 
authorised forms MUST undergo 
assessment by the Faculty Head of 
Research Ethics. Applications 
must be submitted by the tutor 
(copying in the applicant) to 
ADHethics@dmu.ac.uk 


Applicants must not conduct 


tutor (copying in the Design applications research of this type without 
applicant) to DRESC@dmu.ac.uk ; Other School written approval from the Faculty 
ADHethics@dmu.ac.uk to be | applications ADHethics@dmu.ac.uk Head Research Ethics 

logged and filed 


14. Medium Risk Reviewer Outcome — (Assessor Use Only) 


Outcome (tick Y as applicable) 


Approved Rejected DD/MM/YYYY 


Comments/feedback (First Submission): 


All ethical issues successfully considered and addressed 


Section C Project Proposal — expand detail of project: e.g. what do you mean by ‘difficulties’? What do you 
mean by ‘academic English’? 


Section C Purpose/Aims of Project — expand abbreviations for clarity: e.g. what do CALL and EFL stand 
for? 


Section D Ethical Issues — state the ethical issues you are addressing in the bullet points. At present you 
are simply stating your intentions in collecting date. Also, as above, set out your abbreviations in full on 
first mention. You must always do this in your participant information and consent forms. 


Section E - more detail here is needed: e.g. what do you mean by ‘minimising personal risks’ and what form 
of ‘password protected’ devise/s will be used for storage. 


Comments/feedback (Second Submission): 


Approved Rejected xX DD/MM/YYYY 


07/06/19 
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Resubmission notes: I have no issue with the unglossed use of terms such as CALL and EFL; these 
are not arcane vocabulary, and the issues concerning the nature of the ‘difficulties’ ESL/EAL 
students face are, I feel, adequately elucidated in the Purpose/Aims section, where the applicant 
shows that they are investigating the ways in which CALL may aid in the development of the lexico- 
grammatical development of student writing. 


1. APPLICATION FORM 
Section C is acceptable. 


Section 5: There needs to be a clear division between IDENTIFYING Ethical issues and methods of 
MITIGATING them. This is, however, a very low-risk project; no identifiable data (as far as can be 
seen) is being collected, and the student is ensuring Informed Consent and RI ght of Withdrawal 
through PCF and PIS. 


However, | recommend that Section 5 is revised as follows, adopting the following structure: 


1B TRICAL ISSUES: 


This project raises the following ethical issues: 


e collecting data from living human participants 

e storing data in electronic form about living human participants 
e ensuring informed consent 

e informing participants of the right to withdraw at any time. 


(these two issues are more challenging with this sample group, who are not native English 
speakers). 


2. MITIGATION OF ETHICAL ISSUES: 


The data collected will not allow the identification of any named individual; personal data collected 
will consist of [STATE WHAT PERSONAL DATA IS COLLECTED: AGE/SEX/LEVEL OF 
EDUCATION? PLEASE BE AS SPECIFIC AS POSSIBLE]. If an individual participant’s contribution to 
the project is discussed, the participant will be referred to either by code number or pseudonym. 


All data collected, both essay texts and participant information, will be held securely in a password- 
protected computer with access restricted to the researcher and their Supervisor. A backup copy of 
the data will be held on a password protected USB drive with similarly restricted access. 


The data held will be destroyed on completion of the project (date of completion = award of final 
mark for dissertation by the appropriate (Re)Assessment Board). Data storage and management 
will be GDPR-compliant. 


Informed consent and knowledge of the right of withdrawal will be ensured through the use of the 
PCF and PIS (copies attached), translated into Arabic. 
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= DE MONTFORT 
WS UNIVERSITY 
LEICESTER 


PARTICIPANT CONSENT FORM 


Highlighted [areas] of this template must be adapted to suit the needs of your particular study. Edited versions of 
this template must be approved by the applicants Tutor/Supervisor in the first instance. You may wish to consider 
using several versions of the consent form if you are conducting different types of research. 


This document should be dated and given a version number so that when amendments are made it is clear which is 
the correct and most recent version. Please also ensure that highlights in this document are removed after 
completion. One copy of the signed and dated consent form should be given to the participant and one copy retained 
by the researcher to be kept securely on file. 

For resubmitted versions please ensure that you track all changes to clearly identify any amendments to the 
reviewer. 


Title of Research Project: Investigating the Efficacy of Flexible Language Acquisition Project (FLAX) in 
Developing Academic Writing: The Case of Saudi University Students. 


Researcher details: Haifa Alresheedi 


P2510819@my365.dmu.ac.uk 


This agreement is made in regard to the essay-writing exercises which took place on [date] 


Please tick and initial all boxes if you have read and understood the following: 


I confirm that I have read and understood the Participant Information Sheet [version 1] for 
the study above. I have had the opportunity to consider the information, ask questions and 
these have been answered satisfactorily 


a) I agree to my data being anonymised and stored securely. 


I understand that my participation is voluntary. I also understand I am free to withdraw at 
any time - without giving any reason and without there being any negative consequences. I 
can decline to answer any particular question, or questions 


I agreed that non-identifiable data may be written up as a dissertation/thesis. 


I understand that De Montfort University has reviewed and approved this study 


I understand that the data collected during the study has been inspected by a supervisor from 
De Montfort University. I give permission for the Supervisor to have access to my data 


I also acknowledge that if I am being interviewed this data may be transcribed by a third 
party, authorised by the university to undertake such duty. 


I agree to take part in the above research project 


I agree to be contacted by the researcher named above 
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Print name of participant 


Participant contact details (e-mail) 


Participant signature 
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PARTICIPANT INFORMATION SHEET 


Highlighted [areas] of this template must be adapted to suit the needs of your particular study. Edited versions of 
this template must be approved by your Tutor/Supervisor/Line Manager in the first instance. You may wish to 
consider using several versions of the Participant Information Sheet if you are conducting different types of 
research. 


This document should be dated and given a version number so that when amendments are made it is clear 
which is the correct and most recent version. Please also ensure that footnotes in this document are referred to 
and removed after completion. A copy of this document should be given to and retained by the participant. For 

resubmitted versions please ensure that you track all changes to clearly identify any amendments to the 
reviewer. 


Title of Research Project: Investigating the Efficacy of Flexible Language Acquisition Project (FLAX) in Developing 


Academic Writing: The Case of Saudi University Students. ! 


Researcher details: Haifa Alresheedi / p2510819@my365.dmu.ac.uk 2 


The researcher named above would like to invite you/ to take part in a research study. Before you decide 
whether to take part it is important for you to understand why the research is being done and what it will involve. 
Please take time to read the following information carefully to decide whether you wish to take part or not. If you 
wish to, please discuss this with friends and relatives. If there is anything that is not clear or if you would like 
more information, please ask the researcher named above. 


What is the purpose of this study? ° 


The research for this study is being undertaken by the researcher named above who is a Master’s degree 
student in Faculty of Arts, Design and Humanities at De Montfort University, Leicester. 


What does the study involve? 4 


If you agree to participate in this study, we will ask you to take part in a short academic writing course, and write 
an essay before and after the intended intervention, The study will take no longer than six weeks and will take 
place in a university in Saudi Arabia. 


Why have I been chosen? 5 
You have been chosen because you are a Saudi national at the university level. 


Up to thirty other participants will also be chosen to take part in this study. 


' Is the title self-explanatory to a lay person? If not, a simplified title should be used, (acronymic titles can only be 
used if they are accompanied by the full title) 

2 Do not use your personal contact details such as home/mobile phone number, or personal e-mail address 

3 The summary must clearly indicate to the participant the background of the project and its aims and what the 
study is about. If the study is for an educational award i.e. BSc/BA, MSc/MA or MPhil/PhD or if you are an 
academic member of staff this must be stated 

4 You should describe exactly what taking part will involve for the researcher. For example, a one hour interview 
in their own home with questions focusing on a specific theme or responding to a questionnaire or attending a 
focus group. Any invasive procedures must be explained here where applicable and it is also essential to explain 
whether any normal treatment will be withheld for all or part of the study. Set down clearly what you expect the 
potential participant to do and explain exactly what will happen to them during the research study. Please also 
state appropriate realistic length of time. 

5 You should explain here how the participant was chosen to be invited to take part in the study and how many 
other research participants will be studied 
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Do I have to take part? © 


Participation in this study is voluntary and you may ask the researcher questions before agreeing to participate. It 
is up to you to decide whether or not to take part. If you do decide to take part you will be given this information 
sheet to keep and be asked to sign a Consent Form before the study takes place. If you decide to take part you 
are still free to withdraw at any time and without giving a reason. 


| am interested in taking part, what do I do next? 
If you are interested in taking part please contact the researcher named above by the e-mail provided. 
What if | agree to take part and then change my mind? ” 


You can withdraw from the study at any time, without giving a reason. If you wish to withdraw from this study 
please contact the researcher named above. 


Any data collated up to the point of withdrawal will be kept securely on a password protected database and 
securely disposed of following the withdrawal request. 


What are the possible disadvantages and risks of taking part? ® 


While we hope that your experience will be pleasant, you will be required to do multiple tasks both in and outside 
class, which may make you feel uncomfortable. If at any point during the study you feel distressed you can 
choose to remove yourself and the session will end. 


What are the possible benefits of taking part? ° 


The benefits of taking part in this study will enhance your academic writing level to a significant extent, will 
expose you to interaction with cutting-edge technology in language education, and will allow you a chance to 
participate in developing second language teaching research. 


What if something goes wrong? 


If you are harmed by taking part in this research project, there are no special compensation arrangements. If you 
are harmed due to someone’s negligence, then you may have grounds for a legal action but you may incur costs. 
Regardless of this, if you wish to complain, or have any concerns about any aspect of the way you have been 
approached or treated during the course of this study, the normal University complaints mechanisms should be 
available to you. 


What should | do if | have any concerns or complaints? '° 


If you have any concerns or complaints you can initially speak to the researcher, who will acknowledge your 
concerns within ten working days. If no satisfactory outcome is achieved or you wish to make a formal complaint, 
please contact 


6 A Participant Consent Form or Parent/Guardian Consent Form is required if you (the applicant) indicates in the 
ADH Ethics Application Form Issue Checklist that this supporting document template is required. Please refer to 
the ADH Ethics Application Form to ensure that the correct document is used and submit the relevant consent 
form as part of the ethics form. 

’ Participants must be advised that they can choose to withdraw from the study at any time, without being giving 
a reason. It is important to state what will happen to any data collected up to the point of withdrawal. If the data is 
collected anonymously it may not be possible to identify and withdraw. For other data you should state how long 
a participant has to make a request to withdraw data. It is acceptable to inform the participant that data cannot be 
withdrawn providing they consent on that understanding. 

8 One of the disadvantages you should identify is the participant giving up of their time. If interview/focus groups 
questions have the potential to cause upset or raise emotive issues you must be clear about what you will do. 
You should make it clear that the interview will cease so they can gather themselves if they wish. 

° These might include direct benefit to participants, or may not benefit participants personally but will provide 
information which will inform debate or can be used to seek funding for more research. 

10 |f the researcher is an academic member of staff this should be their Line Manager. If the researcher is a 
student this should be their tutor/supervisor. Only DMU addresses should be included. 
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The school of Humanities, De Montfort University, Leicester, LE1 9BH 
Email address: _jie.liu@dmu.ac.uk 
Will taking part in this study be kept confidential? " 


All the information that we collect about you during the course of the research will be necessary to the study and 
will be kept for five years after the study has been completed. All collated data during the course of the research 
will be stored on a password protected database and will be kept strictly confidential. 


Data will be managed by the researcher named above for the duration of the project. Only the researcher and 
Supervisor will have access to the data. The Faculty Head of Research Ethics may also require access to check 
that the study has been conducted in accordance with the approval. 


The Participant will be given an ID code which will be used instead of your name, unless this is contrary to your 
wishes. Any identifiable information you may give will be removed and anonymised. Any published data released 
to a third party will be anonymised so that it cannot be traced back to the Participant. 


You should also be aware that the researcher may be duty bound to pass on information that you provide that 
reveals harm has occurred to a child or other vulnerable individual. 


What will happen to the results of the study? 12 


If you agree to participate in this study, the research will be written up and used in a dissertation and a digital 
copy of this project will be submitted on Turnitin. 


Please note that data [submitted on Turinitin may be shared with third parties. This research may also be used by 
other researchers and regulatory authorities for future research. 


If you require it, you may request a copy of the findings from the researcher named above by e-mail which will be 
submitted to you. 


Who is funding the research? 1% 
No funding body. 
Who has reviewed this study? ‘4 


This study has been reviewed and approved by De Montfort University, Faculty of Arts, Design and Humanities 
Research Ethics Committee. 


Contact for Further Information 


Thank you for volunteering for taking part in the study. For any enquiries please contact the researcher at the e- 
mail address listed above. 


'! Consider anonymising the participants by identifying the participant with an ID number, e.g. Participant A, B, C 
etc. Do not use their initials as this may identify the participant. 

If you are conducting a focus group interview and/or recording a performance which will be available for public 
view you cannot promise confidentiality. 

Duty cannot be imposed on all participants taking part in a focus group. If this is the type of study you are 
conducting you will need to state this in this section. 

You should also consider the possibility that you may require consent from a parent/guardian and that a 
participant may reveal information that a child or other vulnerable person has been or is being harmed. There 
may be a legal, professional or moral requirement for you to reveal that information and this should be included in 
the information sheet. 

'2 It is important to state here who else will have access to this information if it is submitted/deposited on 
Turnitin/DORA. If this is for an educational award and your dissertation/thesis will be submitted on Turnitin, data 
may be shared with third parties. If this is for an academic study your publication/journal may be deposited onto 
DORA which will means that it will be available to other internet users. Please specify which depending on the 
type of study. It is also good practice to indicate that the research may be used by other researchers and 
regulatory authorities for future research. 

'3 This section is only required if the research project is being funded by an external funder or receiving internal 
funding. 

14 Please note that researchers must not conduct their research until they have been granted full ethical approval. 
Do not include names of any individuals who may have reviewed the study. 
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2- Permission letters. 


eS 


ote ao & - 4 
Kingdom of Saudi Arabia wage ‘a Tay 
assim, S tai 


Ministry of Education at a yee acer 
Qassim University (Passiris Sead SeonUs w geese) 23 
Deanship of Educational Services 1c oe 
Arete! GLeiaN Saloc 
a a Alin la & + abst el] (ai) 


: Eguc oll 2 Lae yt 


To: University of Demont Fort, UK 


The Deanship of Educational Services (Preparatory Year 
Program) does not object to fulfill the wish of the student / 
Haifa Mubarak Al-Resheedi who is from Saudi Arabia and is 
studying as a student in University of Demont Fort, UK, in 
collecting data related to her research from the students of the 
summer semester in the Deanship of Educational Services 
(Preparatory Year Program), Qassim University. This letter was 
given to her upon her request. 


The University is accredited By NCAAA <] : US orvizo aszolall - 
(National Commission For Academic Accreditation and Assessment) = cguoyal Sul aaa Sd A39-Hi Gilg! Giral 
May 1. 2013 . April 30. 2017 Frairal Zell! ot 23e Hy CAV Qupl - B+ - F-IP gylo - 1 


Direct:(016) 3801862 ~ Fax:(016) 3801728 - P.O.Box: 6595 - Code: 51452 OVEOY Sty pad) ja p31 — VOMO 29. pe (411) PANVYA sual — (+11) KANATY spcilee 
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-” syn * / 
Kingdom of Saudia Arabia 239 anil ww 


Ministry of Education Ld sElye 
assim Universi ssi RO. Wei 5 


Deanship of Educational Services 
Anertait!) Glerst! Balec 


: (-£) 


To: University of Demont Fort UK 
Dated: 08/08/2019 


Subject: Confirmation Regarding Correction of Exams related to Research Project 
of Haifa Mubarak Al-Rasheedi 

It is confirmed that two English language teachers in the English language Unit of 
the Deanship of Educational Services, Qassim University named: 


1) Mr. Haroon Ayaz, 2) Mr. Shehzad Ahmed have corrected the writing tests with 
regard to Haifa Mubarak Al-Rasheedi’s research project at the University of 


Demont Fort UK. 


Dr. Saleh S. Alhewairini 


Go Loss l4/ danino aaolall 


The University is accredited BY EEC-HES 
val al eis pull yg aie 


{Education Evaluation Commission - Hiegher Education Sector) 
May 4. 2013 - April 30.2020 


Direct:(016)3801862 - Fax : (016)3801728 - P.O.Box : 6595 - Code : 51452 
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wi, 2 | 
Kingdom of Saudia Arabia x9-\ mee a 
Ministry of Education AAT N39 
Qassim University = a 2 — 
Deanship of Educational Services niversily LOR sa, eeeaall Aacl > 
Aretlait! Gleisti salec 


= (--) 


*Egaegl! 1 Glaa,tt 


To: University of Demont Fort UK 
Dated: 08/08/2019 


Subject: Confirmation Regarding Correction of Exams related to Research Project 
of Haifa Mubarak Al-Rasheedi 


It is confirmed that two English language teachers in the English language Unit of 
the Deanship of Educational Services, Qassim University named: 


1) Mr. Haroon Ayaz, 2) Mr. Shehzad Ahmed have corrected the writing tests with 
regard to Haifa Mubarak Al-Rasheedi’s research project at the University of 
Demont Fort UK. 


Dr. Saleh S. Alhewairini 


@ 
The University is accredited BY EEC-HES a Go LesalSi danino doolall 


{Education Evaluation Commission - Hiegher Education Sector) — yloll puloill clbé - piloil! pigai dix 
May 1. 2013 - April 302020 Lal yy f- It quilo- 
Direct:(016)3801862 - Fax : (016)3801728 - P.O.Box : 6595 - Code : 51452 
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LEICESTER 


= DE MONTFORT 
Ss UNIVERSITY 


13 June 2019 
P2510819 


Haifa Alresheedi 
Navigation Street 

Fifth Floor, Apartment 20 
Leicester 

LEI 3UJ 


Dear Sir/Madam, 


Re: Haifa Alresheedi (date of birth: 22/12/91) 
MA English Language Teaching (X14272) 


I'm writing to grant permission for Haifa Alresheedi to collect data for her dissertation from Al Qassim 
University in Saudi Arabia. She will be undertaking voluntary teaching of English Language classes there 
from Tuesday 18th June to Thursday 8th August, and will be doing questionnaires and interviews with 
staff and students at the University. 


Please do not hesitate to contact her supervisor, Susan Barwick (email:!susan.barwick@dmu.ac.uk) if you 


have any queries. 


Thank you. 


Yours sincerely, 
Db 


Lynda McLaughlin 
Postgraduate Coordinator 
Faculty of Arts, Design and Humanities 


Faculty of Art, Design & Humanities, The Gateway, Leicester LEI 9BH. T: (0116) 255 1551 F: (0116) 257 7265 
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4- Lesson plans: 


https://documentcloud.adobe.com/link/track?uri=urn%3Aaaid%3Ascds*3AUS 
%3A76ce2473-eb5b-48a7 -bbe8 -f928ec4841f0 


5- Class materials: 


https://documentcloud.adobe.com/link/track?uri=urn%3Aaaid%3Ascds*3AUS 
%3A1dea067f -3d0c-4757 -9bc2-749b9c50195c 


https://documentcloud.adobe.com/link/track?uri=urn%3Aaaid%3Ascds%3AUS% 
3A6ac82161-a767-4b8d-bb4d-c93a851473d6 


6- Rubric: 


https://documentcloud.adobe.com/link/track?uri=urn%3Aaaid%3Ascds%3AUS% 
3Ad6697780-cle6-4cf2-995c-6cf62cafi8ed 


7- Pretest and posttest for intervention group: 


https: //documentcloud.adobe.com /link/track?uri=urn%3Aaaid%3Ascds 
%3AUS%3A411819f2-702a-4020-a9b7-212616dc0907 


8- Control group: 


https: //documentcloud.adobe.com/link/track?uri=urn%3Aaaid%3Ascds%3AUS%3Ale 
e76992-71e2-417d-9d1f-1laff5cb4465f 


9- Statstic for the intenention group: 
https: //documentcloud.adobe.com /link/track?uri=urn%3Aaaid%3Ascds%3AUS%3Ad2 
Ocecid-c5f9-4384-807c-84f79a5e5640 

10- Statstic for the control group: 


https: //documentcloud.adobe.com/link/track?uri=urn%3Aaaid%3Ascds%3AUS 
%3Aa8856388-ad50-497b-a30b-9506cbd517dc 


11- Output: 


https: //documentcloud.adobe.com/link/track?uri=urn%3Aaaid%3Ascds%3AUS 
%3Aabe70f6d-cecc-40cd-b8e9-8d169a2be099 
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